2026-02-13T18:54:12.753 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T18:54:12.768 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T18:54:12.843 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49760 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '49760' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5319 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial034.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2Ct7nCf388LVzH34pBhn0D9Cf4L7g8FObIYyjgk7Ee8UBP9inMlJW06s/xA7codSMCR0GsfTq1difTHj4fDlU= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-13_16:37:03 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-13T18:54:12.843 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-13T18:54:12.843 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-13T18:54:12.844 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-13T18:54:12.846 INFO:teuthology.task.internal:Checking packages... 2026-02-13T18:54:12.846 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-13T18:54:12.846 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T18:54:12.846 INFO:teuthology.packaging:ref: None 2026-02-13T18:54:12.846 INFO:teuthology.packaging:tag: None 2026-02-13T18:54:12.846 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-13T18:54:12.846 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T18:54:12.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-13T18:54:12.962 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230.gafefaa40 2026-02-13T18:54:12.963 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-13T18:54:12.975 INFO:teuthology.task.internal:no buildpackages task found 2026-02-13T18:54:12.975 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-13T18:54:12.988 INFO:teuthology.task.internal:Saving configuration 2026-02-13T18:54:12.995 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-13T18:54:13.007 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-13T18:54:13.082 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial034.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49760', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 18:50:44.509178', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2Ct7nCf388LVzH34pBhn0D9Cf4L7g8FObIYyjgk7Ee8UBP9inMlJW06s/xA7codSMCR0GsfTq1difTHj4fDlU='} 2026-02-13T18:54:13.082 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-13T18:54:13.093 INFO:teuthology.task.internal:roles: ubuntu@trial034.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-13T18:54:13.093 INFO:teuthology.run_tasks:Running task console_log... 2026-02-13T18:54:13.171 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c9ffb6020>, signals=[15]) 2026-02-13T18:54:13.172 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-13T18:54:13.183 INFO:teuthology.task.internal:Opening connections... 2026-02-13T18:54:13.183 DEBUG:teuthology.task.internal:connecting to ubuntu@trial034.front.sepia.ceph.com 2026-02-13T18:54:13.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial034.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T18:54:13.247 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-13T18:54:13.259 DEBUG:teuthology.orchestra.run.trial034:> uname -m 2026-02-13T18:54:13.322 INFO:teuthology.orchestra.run.trial034.stdout:x86_64 2026-02-13T18:54:13.322 DEBUG:teuthology.orchestra.run.trial034:> cat /etc/os-release 2026-02-13T18:54:13.377 INFO:teuthology.orchestra.run.trial034.stdout:NAME="CentOS Stream" 2026-02-13T18:54:13.377 INFO:teuthology.orchestra.run.trial034.stdout:VERSION="9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:ID="centos" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:ID_LIKE="rhel fedora" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:VERSION_ID="9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:PLATFORM_ID="platform:el9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:ANSI_COLOR="0;31" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:LOGO="fedora-logo-icon" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:HOME_URL="https://centos.org/" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T18:54:13.378 INFO:teuthology.orchestra.run.trial034.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T18:54:13.379 INFO:teuthology.lock.ops:Updating trial034.front.sepia.ceph.com on lock server 2026-02-13T18:54:13.463 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-13T18:54:13.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-13T18:54:13.486 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-13T18:54:13.486 DEBUG:teuthology.orchestra.run.trial034:> test '!' -e /home/ubuntu/cephtest 2026-02-13T18:54:13.501 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-13T18:54:13.512 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-13T18:54:13.512 DEBUG:teuthology.orchestra.run.trial034:> test -z $(ls -A /var/lib/ceph) 2026-02-13T18:54:13.557 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-13T18:54:13.662 INFO:teuthology.run_tasks:Running task kernel... 2026-02-13T18:54:13.701 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-13T18:54:13.701 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-13T18:54:13.701 DEBUG:teuthology.orchestra.run.trial034:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T18:54:13.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T18:54:13.724 DEBUG:teuthology.orchestra.run.trial034:> uname -r 2026-02-13T18:54:13.782 INFO:teuthology.orchestra.run.trial034.stdout:5.14.0-645.el9.x86_64 2026-02-13T18:54:13.782 INFO:teuthology.task.kernel:Running kernel on trial034: 5.14.0-645.el9.x86_64 2026-02-13T18:54:13.782 DEBUG:teuthology.orchestra.run.trial034:> sudo yum install -y kernel 2026-02-13T18:54:15.877 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.9 MB 00:01 2026-02-13T18:54:18.951 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-13T18:54:22.232 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 7.8 MB 00:00 2026-02-13T18:54:23.107 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - Extras packages 130 kB/s | 20 kB 00:00 2026-02-13T18:54:23.742 INFO:teuthology.orchestra.run.trial034.stdout:Extra Packages for Enterprise Linux 36 MB/s | 20 MB 00:00 2026-02-13T18:54:26.981 INFO:teuthology.orchestra.run.trial034.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-13T18:54:28.044 INFO:teuthology.orchestra.run.trial034.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T18:54:28.064 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T18:54:28.065 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:54:28.065 INFO:teuthology.orchestra.run.trial034.stdout: Package Architecture Version Repository Size 2026-02-13T18:54:28.065 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:54:28.065 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T18:54:28.065 INFO:teuthology.orchestra.run.trial034.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:Install 4 Packages 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 88 M 2026-02-13T18:54:28.066 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 135 M 2026-02-13T18:54:28.067 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T18:54:28.320 INFO:teuthology.orchestra.run.trial034.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 3.9 MB/s | 768 kB 00:00 2026-02-13T18:54:28.487 INFO:teuthology.orchestra.run.trial034.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 47 MB/s | 17 MB 00:00 2026-02-13T18:54:28.656 INFO:teuthology.orchestra.run.trial034.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 92 MB/s | 31 MB 00:00 2026-02-13T18:54:28.760 INFO:teuthology.orchestra.run.trial034.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 62 MB/s | 40 MB 00:00 2026-02-13T18:54:28.760 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T18:54:28.760 INFO:teuthology.orchestra.run.trial034.stdout:Total 127 MB/s | 88 MB 00:00 2026-02-13T18:54:28.951 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T18:54:29.151 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T18:54:29.151 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T18:54:29.495 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T18:54:29.495 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T18:54:30.577 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T18:54:30.862 INFO:teuthology.orchestra.run.trial034.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T18:54:30.881 INFO:teuthology.orchestra.run.trial034.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T18:54:32.655 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T18:54:32.684 INFO:teuthology.orchestra.run.trial034.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T18:54:35.712 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T18:54:35.719 INFO:teuthology.orchestra.run.trial034.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:54:38.811 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:55:09.479 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:55:09.500 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:55:09.565 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:55:09.565 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T18:55:09.565 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T18:55:09.565 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.617 INFO:teuthology.orchestra.run.trial034.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.618 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:55:09.618 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T18:55:09.675 DEBUG:teuthology.orchestra.run.trial034:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T18:55:09.700 INFO:teuthology.orchestra.run.trial034.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.701 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T18:55:09.702 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T18:55:09.702 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T18:55:09.702 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T18:55:09.702 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-13T18:55:09.702 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial034.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T18:55:09.702 DEBUG:teuthology.orchestra.run.trial034:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T18:55:09.765 INFO:teuthology.orchestra.run.trial034.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T18:55:09.766 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T18:55:09.767 DEBUG:teuthology.orchestra.run.trial034:> sudo rpm -qi grub2-tools 2026-02-13T18:55:09.835 INFO:teuthology.orchestra.run.trial034.stdout:Name : grub2-tools 2026-02-13T18:55:09.835 INFO:teuthology.orchestra.run.trial034.stdout:Epoch : 1 2026-02-13T18:55:09.835 INFO:teuthology.orchestra.run.trial034.stdout:Version : 2.06 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Release : 118.el9 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Architecture: x86_64 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Group : Unspecified 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Size : 8393909 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:License : GPLv3+ 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T18:55:09.836 INFO:teuthology.orchestra.run.trial034.stdout:Packager : builder@centos.org 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:Vendor : CentOS 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:Summary : Support tools for GRUB. 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:Description : 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:hardware devices. 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:55:09.837 INFO:teuthology.orchestra.run.trial034.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T18:55:09.839 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T18:55:09.839 INFO:teuthology.task.kernel:Updating grub on trial034 to boot 5.14.0-677.el9.x86_64 2026-02-13T18:55:09.839 DEBUG:teuthology.orchestra.run.trial034:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T18:55:09.904 INFO:teuthology.orchestra.run.trial034.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T18:55:09.906 DEBUG:teuthology.orchestra.run.trial034:> sudo command -v grubby && echo yes || echo no 2026-02-13T18:55:09.969 INFO:teuthology.orchestra.run.trial034.stdout:/sbin/grubby 2026-02-13T18:55:09.971 INFO:teuthology.orchestra.run.trial034.stdout:yes 2026-02-13T18:55:09.971 DEBUG:teuthology.orchestra.run.trial034:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T18:55:10.035 INFO:teuthology.orchestra.run.trial034.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T18:55:10.037 DEBUG:teuthology.orchestra.run.trial034:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T18:55:10.111 DEBUG:teuthology.orchestra.run.trial034:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T18:55:10.325 INFO:teuthology.orchestra.run.trial034.stderr:Generating grub configuration file ... 2026-02-13T18:55:10.389 INFO:teuthology.orchestra.run.trial034.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T18:55:10.396 INFO:teuthology.orchestra.run.trial034.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T18:55:10.649 INFO:teuthology.orchestra.run.trial034.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T18:55:10.656 INFO:teuthology.orchestra.run.trial034.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T18:55:10.687 INFO:teuthology.orchestra.run.trial034.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T18:55:10.694 INFO:teuthology.orchestra.run.trial034.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T18:55:10.717 INFO:teuthology.orchestra.run.trial034.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T18:55:10.723 INFO:teuthology.orchestra.run.trial034.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T18:55:10.806 INFO:teuthology.orchestra.run.trial034.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T18:55:10.814 INFO:teuthology.orchestra.run.trial034.stderr:done 2026-02-13T18:55:10.817 DEBUG:teuthology.orchestra.run.trial034:> test -d /sys/firmware/efi 2026-02-13T18:55:10.833 DEBUG:teuthology.orchestra.run.trial034:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T18:55:10.898 INFO:teuthology.orchestra.run.trial034.stdout:/boot/efi/EFI/BOOT 2026-02-13T18:55:10.898 INFO:teuthology.orchestra.run.trial034.stdout:/boot/efi/EFI/centos 2026-02-13T18:55:10.899 DEBUG:teuthology.orchestra.run.trial034:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T18:55:10.963 INFO:teuthology.orchestra.run.trial034.stdout:no 2026-02-13T18:55:10.964 DEBUG:teuthology.orchestra.run.trial034:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T18:55:11.027 INFO:teuthology.orchestra.run.trial034.stdout:yes 2026-02-13T18:55:11.028 DEBUG:teuthology.orchestra.run.trial034:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T18:55:11.091 DEBUG:teuthology.orchestra.run.trial034:> sudo shutdown -r now 2026-02-13T18:55:41.164 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-13T18:55:41.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial034.front.sepia.ceph.com' 2026-02-13T18:55:41.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial034.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T18:56:41.220 DEBUG:teuthology.orchestra.remote:timed out 2026-02-13T18:56:50.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial034.front.sepia.ceph.com' 2026-02-13T18:56:50.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial034.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T18:56:50.368 DEBUG:teuthology.orchestra.run.trial034:> true 2026-02-13T18:56:50.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial034.front.sepia.ceph.com' 2026-02-13T18:56:50.517 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-13T18:56:50.518 DEBUG:teuthology.orchestra.run.trial034:> uname -r 2026-02-13T18:56:50.573 INFO:teuthology.orchestra.run.trial034.stdout:5.14.0-677.el9.x86_64 2026-02-13T18:56:50.574 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T18:56:50.574 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T18:56:50.574 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T18:56:51.575 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-13T18:56:51.575 DEBUG:teuthology.orchestra.run.trial034:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-13T18:56:51.605 INFO:teuthology.orchestra.run.trial034.stdout:ttyS1 2026-02-13T18:56:51.635 DEBUG:teuthology.parallel:result is None 2026-02-13T18:56:51.635 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-13T18:56:51.650 INFO:teuthology.task.internal:Creating test directory... 2026-02-13T18:56:51.650 DEBUG:teuthology.orchestra.run.trial034:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T18:56:51.691 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-13T18:56:51.703 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-13T18:56:51.717 INFO:teuthology.task.internal:Creating archive directory... 2026-02-13T18:56:51.717 DEBUG:teuthology.orchestra.run.trial034:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T18:56:51.755 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-13T18:56:51.767 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-13T18:56:51.767 DEBUG:teuthology.orchestra.run.trial034:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T18:56:51.815 INFO:teuthology.orchestra.run.trial034.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T18:56:51.825 INFO:teuthology.orchestra.run.trial034.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T18:56:51.826 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-13T18:56:51.838 INFO:teuthology.task.internal:Configuring sudo... 2026-02-13T18:56:51.839 DEBUG:teuthology.orchestra.run.trial034:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T18:56:51.891 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-13T18:56:51.908 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-13T18:56:51.908 DEBUG:teuthology.orchestra.run.trial034:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T18:56:51.948 DEBUG:teuthology.orchestra.run.trial034:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T18:56:52.110 DEBUG:teuthology.orchestra.run.trial034:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T18:56:52.135 DEBUG:teuthology.orchestra.run.trial034:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T18:56:52.287 DEBUG:teuthology.orchestra.run.trial034:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T18:56:52.311 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T18:56:52.311 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T18:56:52.377 DEBUG:teuthology.orchestra.run.trial034:> sudo service rsyslog restart 2026-02-13T18:56:52.444 INFO:teuthology.orchestra.run.trial034.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T18:56:52.838 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-13T18:56:52.850 INFO:teuthology.task.internal:Starting timer... 2026-02-13T18:56:52.850 INFO:teuthology.run_tasks:Running task pcp... 2026-02-13T18:56:52.873 INFO:teuthology.run_tasks:Running task selinux... 2026-02-13T18:56:52.891 DEBUG:teuthology.orchestra.run.trial034:> sudo service auditd rotate 2026-02-13T18:56:52.921 INFO:teuthology.orchestra.run.trial034.stdout:Rotating logs: 2026-02-13T18:56:52.923 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-13T18:56:52.923 DEBUG:teuthology.orchestra.run.trial034:> /usr/sbin/getenforce 2026-02-13T18:56:52.981 INFO:teuthology.orchestra.run.trial034.stdout:Permissive 2026-02-13T18:56:52.983 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial034.front.sepia.ceph.com': 'permissive'} 2026-02-13T18:56:52.983 DEBUG:teuthology.orchestra.run.trial034:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T18:56:53.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T18:56:53.048 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-13T18:56:53.049 DEBUG:teuthology.orchestra.run.trial034:> sudo /usr/sbin/setenforce permissive 2026-02-13T18:56:53.113 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-13T18:56:53.127 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-13T18:56:53.131 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-13T18:56:53.276 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-13T18:56:53.282 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-13T18:56:53.283 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial034.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-13T18:59:40.090 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial034.front.sepia.ceph.com')] 2026-02-13T18:59:40.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial034.front.sepia.ceph.com' 2026-02-13T18:59:40.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial034.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T18:59:40.205 DEBUG:teuthology.orchestra.run.trial034:> true 2026-02-13T18:59:40.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial034.front.sepia.ceph.com' 2026-02-13T18:59:40.274 INFO:teuthology.run_tasks:Running task clock... 2026-02-13T18:59:40.290 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-13T18:59:40.290 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T18:59:40.290 DEBUG:teuthology.orchestra.run.trial034:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T18:59:40.341 INFO:teuthology.orchestra.run.trial034.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T18:59:40.353 INFO:teuthology.orchestra.run.trial034.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T18:59:40.374 INFO:teuthology.orchestra.run.trial034.stderr:sudo: ntpd: command not found 2026-02-13T18:59:40.384 INFO:teuthology.orchestra.run.trial034.stdout:506 Cannot talk to daemon 2026-02-13T18:59:40.396 INFO:teuthology.orchestra.run.trial034.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T18:59:40.409 INFO:teuthology.orchestra.run.trial034.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T18:59:40.460 INFO:teuthology.orchestra.run.trial034.stderr:bash: line 1: ntpq: command not found 2026-02-13T18:59:40.482 INFO:teuthology.orchestra.run.trial034.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T18:59:40.483 INFO:teuthology.orchestra.run.trial034.stdout:=============================================================================== 2026-02-13T18:59:40.483 INFO:teuthology.orchestra.run.trial034.stdout:^? chi3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T18:59:40.483 INFO:teuthology.orchestra.run.trial034.stdout:^? ntp.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T18:59:40.483 INFO:teuthology.orchestra.run.trial034.stdout:^? time.trtnw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T18:59:40.483 INFO:teuthology.orchestra.run.trial034.stdout:^? as393746.mci.trtnw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T18:59:40.484 INFO:teuthology.run_tasks:Running task install... 2026-02-13T18:59:40.498 DEBUG:teuthology.task.install:project ceph 2026-02-13T18:59:40.499 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'}} 2026-02-13T18:59:40.499 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'} 2026-02-13T18:59:40.499 INFO:teuthology.task.install:Using flavor: default 2026-02-13T18:59:40.509 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-13T18:59:40.510 INFO:teuthology.task.install:extra packages: [] 2026-02-13T18:59:40.510 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6', 'tag': None, 'wait_for_package': False} 2026-02-13T18:59:40.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T18:59:40.669 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/ 2026-02-13T18:59:40.669 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5230.gafefaa40 2026-02-13T18:59:40.812 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T18:59:40.812 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T18:59:40.812 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T18:59:40.838 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-13T18:59:40.838 DEBUG:teuthology.orchestra.run.trial034:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/afefaa40503a4bf024e96f97dd7f852b75f00ad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T18:59:40.905 DEBUG:teuthology.orchestra.run.trial034:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T18:59:40.977 DEBUG:teuthology.orchestra.run.trial034:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T18:59:41.043 INFO:teuthology.orchestra.run.trial034.stdout:check_obsoletes = 1 2026-02-13T18:59:41.044 DEBUG:teuthology.orchestra.run.trial034:> sudo yum clean all 2026-02-13T18:59:41.194 INFO:teuthology.orchestra.run.trial034.stdout:41 files removed 2026-02-13T18:59:41.209 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-radosgw 2026-02-13T18:59:41.480 INFO:teuthology.orchestra.run.trial034.stdout:ceph packages for x86_64 727 kB/s | 91 kB 00:00 2026-02-13T18:59:41.623 INFO:teuthology.orchestra.run.trial034.stdout:ceph noarch packages 201 kB/s | 24 kB 00:00 2026-02-13T18:59:41.752 INFO:teuthology.orchestra.run.trial034.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-13T18:59:42.817 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.9 MB 00:01 2026-02-13T18:59:46.204 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 27 MB 00:02 2026-02-13T18:59:48.956 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - CRB 20 MB/s | 7.8 MB 00:00 2026-02-13T18:59:50.160 INFO:teuthology.orchestra.run.trial034.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 20 kB 00:00 2026-02-13T18:59:50.744 INFO:teuthology.orchestra.run.trial034.stdout:Extra Packages for Enterprise Linux 40 MB/s | 20 MB 00:00 2026-02-13T18:59:54.070 INFO:teuthology.orchestra.run.trial034.stdout:lab-extras 1.2 MB/s | 50 kB 00:00 2026-02-13T18:59:55.364 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T18:59:55.366 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:59:55.366 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T18:59:55.366 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: ceph-radosgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 13 M 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout:Upgrading: 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: librados2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 3.5 M 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: librbd1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.8 M 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: ceph-base x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.2 M 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 24 M 2026-02-13T18:59:55.367 INFO:teuthology.orchestra.run.trial034.stdout: ceph-selinux x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 909 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: libradosstriper1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 248 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T18:59:55.368 INFO:teuthology.orchestra.run.trial034.stdout: librgw2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 M 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-ceph-argparse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 45 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 185 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-cephfs x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 203 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-rados x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 324 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-rbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 305 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: python3-rgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 100 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T18:59:55.369 INFO:teuthology.orchestra.run.trial034.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout:Installing weak dependencies: 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout:Install 32 Packages 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout:Upgrade 2 Packages 2026-02-13T18:59:55.370 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T18:59:55.371 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 67 M 2026-02-13T18:59:55.371 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T18:59:55.866 INFO:teuthology.orchestra.run.trial034.stdout:(1/34): ceph-base-20.3.0-5230.gafefaa40.el9.x86 17 MB/s | 5.2 MB 00:00 2026-02-13T18:59:55.906 INFO:teuthology.orchestra.run.trial034.stdout:(2/34): ceph-selinux-20.3.0-5230.gafefaa40.el9. 629 kB/s | 25 kB 00:00 2026-02-13T18:59:55.966 INFO:teuthology.orchestra.run.trial034.stdout:(3/34): libcephfs2-20.3.0-5230.gafefaa40.el9.x8 15 MB/s | 909 kB 00:00 2026-02-13T18:59:56.014 INFO:teuthology.orchestra.run.trial034.stdout:(4/34): libradosstriper1-20.3.0-5230.gafefaa40. 5.1 MB/s | 248 kB 00:00 2026-02-13T18:59:56.257 INFO:teuthology.orchestra.run.trial034.stdout:(5/34): ceph-radosgw-20.3.0-5230.gafefaa40.el9. 19 MB/s | 13 MB 00:00 2026-02-13T18:59:56.291 INFO:teuthology.orchestra.run.trial034.stdout:(6/34): python3-ceph-argparse-20.3.0-5230.gafef 1.3 MB/s | 45 kB 00:00 2026-02-13T18:59:56.325 INFO:teuthology.orchestra.run.trial034.stdout:(7/34): librgw2-20.3.0-5230.gafefaa40.el9.x86_6 21 MB/s | 6.5 MB 00:00 2026-02-13T18:59:56.327 INFO:teuthology.orchestra.run.trial034.stdout:(8/34): python3-ceph-common-20.3.0-5230.gafefaa 5.0 MB/s | 185 kB 00:00 2026-02-13T18:59:56.346 INFO:teuthology.orchestra.run.trial034.stdout:(9/34): python3-cephfs-20.3.0-5230.gafefaa40.el 9.7 MB/s | 203 kB 00:00 2026-02-13T18:59:56.353 INFO:teuthology.orchestra.run.trial034.stdout:(10/34): python3-rados-20.3.0-5230.gafefaa40.el 12 MB/s | 324 kB 00:00 2026-02-13T18:59:56.373 INFO:teuthology.orchestra.run.trial034.stdout:(11/34): python3-rbd-20.3.0-5230.gafefaa40.el9. 11 MB/s | 305 kB 00:00 2026-02-13T18:59:56.375 INFO:teuthology.orchestra.run.trial034.stdout:(12/34): python3-rgw-20.3.0-5230.gafefaa40.el9. 4.5 MB/s | 100 kB 00:00 2026-02-13T18:59:56.559 INFO:teuthology.orchestra.run.trial034.stdout:(13/34): ceph-common-20.3.0-5230.gafefaa40.el9. 24 MB/s | 24 MB 00:00 2026-02-13T18:59:56.602 INFO:teuthology.orchestra.run.trial034.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 2.4 MB/s | 104 kB 00:00 2026-02-13T18:59:56.618 INFO:teuthology.orchestra.run.trial034.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 9.7 MB/s | 164 kB 00:00 2026-02-13T18:59:56.629 INFO:teuthology.orchestra.run.trial034.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 15 MB/s | 160 kB 00:00 2026-02-13T18:59:56.638 INFO:teuthology.orchestra.run.trial034.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 5.3 MB/s | 45 kB 00:00 2026-02-13T18:59:56.658 INFO:teuthology.orchestra.run.trial034.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 32 MB/s | 662 kB 00:00 2026-02-13T18:59:56.668 INFO:teuthology.orchestra.run.trial034.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 28 MB/s | 292 kB 00:00 2026-02-13T18:59:56.678 INFO:teuthology.orchestra.run.trial034.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 24 MB/s | 240 kB 00:00 2026-02-13T18:59:56.687 INFO:teuthology.orchestra.run.trial034.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 4.4 MB/s | 37 kB 00:00 2026-02-13T18:59:56.695 INFO:teuthology.orchestra.run.trial034.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.7 MB/s | 66 kB 00:00 2026-02-13T18:59:56.890 INFO:teuthology.orchestra.run.trial034.stdout:(23/34): mailcap-2.1.49-5.el9.noarch.rpm 64 kB/s | 33 kB 00:00 2026-02-13T18:59:56.904 INFO:teuthology.orchestra.run.trial034.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.4 MB/s | 308 kB 00:00 2026-02-13T18:59:56.931 INFO:teuthology.orchestra.run.trial034.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 897 kB/s | 25 kB 00:00 2026-02-13T18:59:56.961 INFO:teuthology.orchestra.run.trial034.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2026-02-13T18:59:56.992 INFO:teuthology.orchestra.run.trial034.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 2.2 MB/s | 67 kB 00:00 2026-02-13T18:59:57.057 INFO:teuthology.orchestra.run.trial034.stdout:(28/34): pciutils-3.7.0-7.el9.x86_64.rpm 137 kB/s | 93 kB 00:00 2026-02-13T18:59:57.068 INFO:teuthology.orchestra.run.trial034.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2026-02-13T18:59:57.132 INFO:teuthology.orchestra.run.trial034.stdout:(30/34): thrift-0.15.0-4.el9.x86_64.rpm 25 MB/s | 1.6 MB 00:00 2026-02-13T18:59:57.200 INFO:teuthology.orchestra.run.trial034.stdout:(31/34): libarrow-9.0.0-15.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2026-02-13T18:59:57.214 INFO:teuthology.orchestra.run.trial034.stdout:(32/34): librados2-20.3.0-5230.gafefaa40.el9.x8 43 MB/s | 3.5 MB 00:00 2026-02-13T18:59:57.243 INFO:teuthology.orchestra.run.trial034.stdout:(33/34): re2-20211101-20.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2026-02-13T18:59:57.274 INFO:teuthology.orchestra.run.trial034.stdout:(34/34): librbd1-20.3.0-5230.gafefaa40.el9.x86_ 38 MB/s | 2.8 MB 00:00 2026-02-13T18:59:57.279 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T18:59:57.279 INFO:teuthology.orchestra.run.trial034.stdout:Total 35 MB/s | 67 MB 00:01 2026-02-13T18:59:57.372 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T18:59:57.389 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T18:59:57.389 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T18:59:57.647 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T18:59:57.648 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T18:59:57.946 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T18:59:58.107 INFO:teuthology.orchestra.run.trial034.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-13T18:59:58.111 INFO:teuthology.orchestra.run.trial034.stdout: Upgrading : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T18:59:58.141 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T18:59:58.179 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-13T18:59:58.181 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T18:59:58.191 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T18:59:58.199 INFO:teuthology.orchestra.run.trial034.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-13T18:59:58.201 INFO:teuthology.orchestra.run.trial034.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-13T18:59:58.204 INFO:teuthology.orchestra.run.trial034.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-13T18:59:58.208 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-13T18:59:58.220 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 9/36 2026-02-13T18:59:58.231 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-13T18:59:58.232 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-13T18:59:58.254 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-13T18:59:58.258 INFO:teuthology.orchestra.run.trial034.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-13T18:59:58.288 INFO:teuthology.orchestra.run.trial034.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-13T18:59:58.291 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-13T18:59:58.297 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-13T18:59:58.300 INFO:teuthology.orchestra.run.trial034.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-13T18:59:58.304 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-13T18:59:58.433 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-13T18:59:58.434 INFO:teuthology.orchestra.run.trial034.stdout: Upgrading : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-13T18:59:58.462 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-13T18:59:58.465 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 20/36 2026-02-13T18:59:58.471 INFO:teuthology.orchestra.run.trial034.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-13T18:59:58.773 INFO:teuthology.orchestra.run.trial034.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-13T18:59:58.774 INFO:teuthology.orchestra.run.trial034.stdout: Installing : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-13T18:59:58.792 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-13T18:59:58.795 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 24/36 2026-02-13T18:59:58.800 INFO:teuthology.orchestra.run.trial034.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-13T18:59:58.803 INFO:teuthology.orchestra.run.trial034.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-13T18:59:58.803 INFO:teuthology.orchestra.run.trial034.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T18:59:58.823 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T18:59:58.826 INFO:teuthology.orchestra.run.trial034.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-13T18:59:58.861 INFO:teuthology.orchestra.run.trial034.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-13T18:59:58.863 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 30/36 2026-02-13T19:00:00.008 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:00:00.011 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:00:00.286 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-13T19:00:00.289 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-13T19:00:00.335 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-13T19:00:00.335 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T19:00:00.335 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T19:00:00.335 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:00.339 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /sys 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /proc 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /mnt 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /var/tmp 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /home 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /root 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout:skipping the directory /tmp 2026-02-13T19:00:04.869 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:05.473 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T19:00:05.498 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:05.500 INFO:teuthology.orchestra.run.trial034.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T19:00:05.512 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T19:00:05.513 INFO:teuthology.orchestra.run.trial034.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:00:05.569 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:00:05.569 INFO:teuthology.orchestra.run.trial034.stdout:Creating group 'qat' with GID 992. 2026-02-13T19:00:05.569 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:05.857 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 1/36 2026-02-13T19:00:05.857 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 5/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 6/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 7/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 8/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 9/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 11/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 12/36 2026-02-13T19:00:05.858 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 13/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-13T19:00:05.859 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-13T19:00:05.860 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 35/36 2026-02-13T19:00:05.916 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T19:00:05.916 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout:Upgraded: 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T19:00:05.917 INFO:teuthology.orchestra.run.trial034.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T19:00:05.918 INFO:teuthology.orchestra.run.trial034.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T19:00:05.919 INFO:teuthology.orchestra.run.trial034.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T19:00:05.920 INFO:teuthology.orchestra.run.trial034.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T19:00:05.920 INFO:teuthology.orchestra.run.trial034.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T19:00:05.920 INFO:teuthology.orchestra.run.trial034.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T19:00:05.920 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:05.920 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:05.956 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-test 2026-02-13T19:00:06.255 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:12 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout: ceph-test x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 78 M 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:06.337 INFO:teuthology.orchestra.run.trial034.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: libcephsqlite x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 159 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout:Install 10 Packages 2026-02-13T19:00:06.338 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:06.339 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 82 M 2026-02-13T19:00:06.339 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 339 M 2026-02-13T19:00:06.339 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:06.943 INFO:teuthology.orchestra.run.trial034.stdout:(1/10): libcephsqlite-20.3.0-5230.gafefaa40.el9 1.6 MB/s | 159 kB 00:00 2026-02-13T19:00:06.985 INFO:teuthology.orchestra.run.trial034.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 544 kB/s | 23 kB 00:00 2026-02-13T19:00:07.002 INFO:teuthology.orchestra.run.trial034.stdout:(3/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2026-02-13T19:00:07.028 INFO:teuthology.orchestra.run.trial034.stdout:(4/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 2.1 MB/s | 94 kB 00:00 2026-02-13T19:00:07.368 INFO:teuthology.orchestra.run.trial034.stdout:(5/10): libxslt-1.1.34-12.el9.x86_64.rpm 636 kB/s | 233 kB 00:00 2026-02-13T19:00:07.470 INFO:teuthology.orchestra.run.trial034.stdout:(6/10): protobuf-3.14.0-17.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2026-02-13T19:00:07.711 INFO:teuthology.orchestra.run.trial034.stdout:(7/10): socat-1.7.4.1-8.el9.x86_64.rpm 885 kB/s | 303 kB 00:00 2026-02-13T19:00:07.717 INFO:teuthology.orchestra.run.trial034.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 258 kB/s | 64 kB 00:00 2026-02-13T19:00:07.891 INFO:teuthology.orchestra.run.trial034.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 678 kB/s | 122 kB 00:00 2026-02-13T19:00:08.396 INFO:teuthology.orchestra.run.trial034.stdout:(10/10): ceph-test-20.3.0-5230.gafefaa40.el9.x8 51 MB/s | 78 MB 00:01 2026-02-13T19:00:08.397 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:08.397 INFO:teuthology.orchestra.run.trial034.stdout:Total 40 MB/s | 82 MB 00:02 2026-02-13T19:00:08.460 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:08.471 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:08.471 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:08.753 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:08.754 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:09.061 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:09.069 INFO:teuthology.orchestra.run.trial034.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-13T19:00:09.078 INFO:teuthology.orchestra.run.trial034.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-13T19:00:09.083 INFO:teuthology.orchestra.run.trial034.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-13T19:00:09.087 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-13T19:00:09.092 INFO:teuthology.orchestra.run.trial034.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-13T19:00:09.094 INFO:teuthology.orchestra.run.trial034.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-13T19:00:09.103 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-13T19:00:09.111 INFO:teuthology.orchestra.run.trial034.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-13T19:00:09.112 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-13T19:00:12.550 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-13T19:00:12.567 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-13T19:00:12.883 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-13T19:00:12.883 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 1/10 2026-02-13T19:00:12.883 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2/10 2026-02-13T19:00:12.883 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-13T19:00:12.884 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-13T19:00:12.936 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T19:00:12.937 INFO:teuthology.orchestra.run.trial034.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T19:00:12.938 INFO:teuthology.orchestra.run.trial034.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T19:00:12.938 INFO:teuthology.orchestra.run.trial034.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-13T19:00:12.938 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:12.938 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:12.975 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph 2026-02-13T19:00:13.270 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:19 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:13.354 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:13.355 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: ceph x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 k 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mds x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.4 M 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 936 k 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 297 k 2026-02-13T19:00:13.356 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mon x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.0 M 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 166 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd-classic x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 16 M 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd-crimson x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 19 M 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T19:00:13.357 INFO:teuthology.orchestra.run.trial034.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-collections 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T19:00:13.358 INFO:teuthology.orchestra.run.trial034.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T19:00:13.359 INFO:teuthology.orchestra.run.trial034.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout:Install 42 Packages 2026-02-13T19:00:13.360 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:13.361 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 48 M 2026-02-13T19:00:13.361 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 167 M 2026-02-13T19:00:13.361 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:13.698 INFO:teuthology.orchestra.run.trial034.stdout:(1/42): ceph-20.3.0-5230.gafefaa40.el9.x86_64.r 129 kB/s | 6.5 kB 00:00 2026-02-13T19:00:13.784 INFO:teuthology.orchestra.run.trial034.stdout:(2/42): ceph-mgr-20.3.0-5230.gafefaa40.el9.x86_ 6.7 MB/s | 936 kB 00:00 2026-02-13T19:00:13.802 INFO:teuthology.orchestra.run.trial034.stdout:(3/42): ceph-mds-20.3.0-5230.gafefaa40.el9.x86_ 15 MB/s | 2.4 MB 00:00 2026-02-13T19:00:13.807 INFO:teuthology.orchestra.run.trial034.stdout:(4/42): ceph-osd-20.3.0-5230.gafefaa40.el9.x86_ 6.9 MB/s | 166 kB 00:00 2026-02-13T19:00:14.026 INFO:teuthology.orchestra.run.trial034.stdout:(5/42): ceph-mon-20.3.0-5230.gafefaa40.el9.x86_ 15 MB/s | 5.0 MB 00:00 2026-02-13T19:00:14.078 INFO:teuthology.orchestra.run.trial034.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5230.gafef 5.5 MB/s | 297 kB 00:00 2026-02-13T19:00:14.335 INFO:teuthology.orchestra.run.trial034.stdout:(7/42): ceph-osd-classic-20.3.0-5230.gafefaa40. 29 MB/s | 16 MB 00:00 2026-02-13T19:00:14.496 INFO:teuthology.orchestra.run.trial034.stdout:(8/42): ceph-osd-crimson-20.3.0-5230.gafefaa40. 28 MB/s | 19 MB 00:00 2026-02-13T19:00:15.899 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] ledmon-libs-1.1.0-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/ledmon-libs-1.1.0-3.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:15.899 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] libconfig-1.7.2-9.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libconfig-1.7.2-9.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:15.899 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:19.102 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] ledmon-libs-1.1.0-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/ledmon-libs-1.1.0-3.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:19.102 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] libconfig-1.7.2-9.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libconfig-1.7.2-9.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:19.102 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:19.334 INFO:teuthology.orchestra.run.trial034.stdout:(9/42): libconfig-1.7.2-9.el9.x86_64.rpm 14 kB/s | 72 kB 00:04 2026-02-13T19:00:19.340 INFO:teuthology.orchestra.run.trial034.stdout:(10/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 7.7 kB/s | 40 kB 00:05 2026-02-13T19:00:19.432 INFO:teuthology.orchestra.run.trial034.stdout:(11/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 51 kB/s | 253 kB 00:04 2026-02-13T19:00:22.043 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-cryptography-36.0.1-5.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cryptography-36.0.1-5.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:22.043 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-ply-3.11-14.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-ply-3.11-14.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:22.044 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-pycparser-2.20-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.20-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:22.316 INFO:teuthology.orchestra.run.trial034.stdout:(12/42): python3-ply-3.11-14.el9.noarch.rpm 36 kB/s | 106 kB 00:02 2026-02-13T19:00:22.323 INFO:teuthology.orchestra.run.trial034.stdout:(13/42): python3-pycparser-2.20-6.el9.noarch.rp 47 kB/s | 135 kB 00:02 2026-02-13T19:00:22.462 INFO:teuthology.orchestra.run.trial034.stdout:(14/42): python3-cryptography-36.0.1-5.el9.x86_ 408 kB/s | 1.2 MB 00:03 2026-02-13T19:00:22.594 INFO:teuthology.orchestra.run.trial034.stdout:(15/42): python3-pyparsing-2.4.7-9.el9.noarch.r 541 kB/s | 150 kB 00:00 2026-02-13T19:00:22.602 INFO:teuthology.orchestra.run.trial034.stdout:(16/42): python3-requests-2.25.1-10.el9.noarch. 455 kB/s | 126 kB 00:00 2026-02-13T19:00:22.778 INFO:teuthology.orchestra.run.trial034.stdout:(17/42): python3-urllib3-1.26.5-7.el9.noarch.rp 688 kB/s | 218 kB 00:00 2026-02-13T19:00:22.872 INFO:teuthology.orchestra.run.trial034.stdout:(18/42): unzip-6.0-59.el9.x86_64.rpm 654 kB/s | 182 kB 00:00 2026-02-13T19:00:22.928 INFO:teuthology.orchestra.run.trial034.stdout:(19/42): zip-3.0-35.el9.x86_64.rpm 816 kB/s | 266 kB 00:00 2026-02-13T19:00:25.130 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] libstoragemgmt-1.10.1-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libstoragemgmt-1.10.1-1.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:25.131 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] lua-5.4.4-4.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lua-5.4.4-4.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:25.131 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-libstoragemgmt-1.10.1-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-libstoragemgmt-1.10.1-1.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 80: No route to host] 2026-02-13T19:00:28.332 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] libstoragemgmt-1.10.1-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libstoragemgmt-1.10.1-1.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:28.333 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] lua-5.4.4-4.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lua-5.4.4-4.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:28.333 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-libstoragemgmt-1.10.1-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-libstoragemgmt-1.10.1-1.el9.x86_64.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:28.823 INFO:teuthology.orchestra.run.trial034.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 30 kB/s | 177 kB 00:05 2026-02-13T19:00:28.834 INFO:teuthology.orchestra.run.trial034.stdout:(21/42): lua-5.4.4-4.el9.x86_64.rpm 32 kB/s | 188 kB 00:05 2026-02-13T19:00:28.860 INFO:teuthology.orchestra.run.trial034.stdout:(22/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 41 kB/s | 246 kB 00:06 2026-02-13T19:00:28.913 INFO:teuthology.orchestra.run.trial034.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 423 kB/s | 22 kB 00:00 2026-02-13T19:00:29.081 INFO:teuthology.orchestra.run.trial034.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 906 kB/s | 151 kB 00:00 2026-02-13T19:00:29.106 INFO:teuthology.orchestra.run.trial034.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 1.1 MB/s | 29 kB 00:00 2026-02-13T19:00:29.138 INFO:teuthology.orchestra.run.trial034.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 1.9 MB/s | 60 kB 00:00 2026-02-13T19:00:29.165 INFO:teuthology.orchestra.run.trial034.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.6 MB/s | 43 kB 00:00 2026-02-13T19:00:29.206 INFO:teuthology.orchestra.run.trial034.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 4.2 MB/s | 173 kB 00:00 2026-02-13T19:00:29.257 INFO:teuthology.orchestra.run.trial034.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 6.8 MB/s | 358 kB 00:00 2026-02-13T19:00:29.281 INFO:teuthology.orchestra.run.trial034.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 453 kB/s | 11 kB 00:00 2026-02-13T19:00:29.305 INFO:teuthology.orchestra.run.trial034.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 744 kB/s | 18 kB 00:00 2026-02-13T19:00:29.328 INFO:teuthology.orchestra.run.trial034.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 979 kB/s | 23 kB 00:00 2026-02-13T19:00:29.352 INFO:teuthology.orchestra.run.trial034.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 827 kB/s | 20 kB 00:00 2026-02-13T19:00:29.378 INFO:teuthology.orchestra.run.trial034.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 773 kB/s | 19 kB 00:00 2026-02-13T19:00:29.401 INFO:teuthology.orchestra.run.trial034.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 1.1 MB/s | 26 kB 00:00 2026-02-13T19:00:29.426 INFO:teuthology.orchestra.run.trial034.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2026-02-13T19:00:29.450 INFO:teuthology.orchestra.run.trial034.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 701 kB/s | 16 kB 00:00 2026-02-13T19:00:29.475 INFO:teuthology.orchestra.run.trial034.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.4 MB/s | 90 kB 00:00 2026-02-13T19:00:29.500 INFO:teuthology.orchestra.run.trial034.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2026-02-13T19:00:29.523 INFO:teuthology.orchestra.run.trial034.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 852 kB/s | 20 kB 00:00 2026-02-13T19:00:31.263 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:31.263 INFO:teuthology.orchestra.run.trial034.stdout:[MIRROR] python3-toml-0.10.2-6.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-toml-0.10.2-6.el9.noarch.rpm [Failed to connect to ftp-osl.osuosl.org port 443: No route to host] 2026-02-13T19:00:31.357 INFO:teuthology.orchestra.run.trial034.stdout:(41/42): python3-toml-0.10.2-6.el9.noarch.rpm 16 kB/s | 42 kB 00:02 2026-02-13T19:00:31.363 INFO:teuthology.orchestra.run.trial034.stdout:(42/42): python3-packaging-20.9-5.el9.noarch.rp 30 kB/s | 77 kB 00:02 2026-02-13T19:00:31.366 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:31.366 INFO:teuthology.orchestra.run.trial034.stdout:Total 2.7 MB/s | 48 MB 00:18 2026-02-13T19:00:31.498 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:31.517 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:31.517 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:31.712 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:31.712 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:31.940 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:31.942 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-13T19:00:31.946 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-13T19:00:31.951 INFO:teuthology.orchestra.run.trial034.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-13T19:00:31.953 INFO:teuthology.orchestra.run.trial034.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-13T19:00:31.959 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-13T19:00:31.982 INFO:teuthology.orchestra.run.trial034.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-13T19:00:31.986 INFO:teuthology.orchestra.run.trial034.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-13T19:00:31.990 INFO:teuthology.orchestra.run.trial034.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-13T19:00:31.995 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-13T19:00:32.001 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-13T19:00:32.005 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-13T19:00:32.011 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-13T19:00:32.016 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-13T19:00:32.019 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-13T19:00:32.025 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-13T19:00:32.028 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-13T19:00:32.035 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-13T19:00:32.039 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-13T19:00:32.062 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-13T19:00:32.074 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-13T19:00:32.080 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-13T19:00:32.089 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-13T19:00:32.096 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-13T19:00:32.109 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-13T19:00:32.124 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-13T19:00:32.200 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-13T19:00:32.208 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-13T19:00:32.231 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-13T19:00:32.261 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-13T19:00:32.267 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-13T19:00:32.318 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-13T19:00:32.359 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 32/42 2026-02-13T19:00:32.360 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-13T19:00:32.383 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-13T19:00:32.384 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:32.384 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T19:00:32.384 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T19:00:32.384 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T19:00:32.384 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:32.388 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-13T19:00:32.406 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:00:32.406 INFO:teuthology.orchestra.run.trial034.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T19:00:32.406 INFO:teuthology.orchestra.run.trial034.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T19:00:32.406 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:32.415 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:00:32.438 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-13T19:00:32.439 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T19:00:32.439 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:33.069 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-13T19:00:33.825 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 37/42 2026-02-13T19:00:33.834 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 38/42 2026-02-13T19:00:33.836 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-13T19:00:33.861 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-13T19:00:33.861 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:33.862 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T19:00:33.862 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T19:00:33.862 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T19:00:33.862 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:34.074 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-13T19:00:34.095 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-13T19:00:34.096 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:34.096 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T19:00:34.096 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T19:00:34.096 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T19:00:34.096 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:34.192 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T19:00:34.213 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:34.214 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-13T19:00:34.220 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-13T19:00:34.243 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-13T19:00:34.633 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-13T19:00:34.633 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 1/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 3/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 4/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 5/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 6/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 7/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 8/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-13T19:00:34.634 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-13T19:00:34.635 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-13T19:00:34.636 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-13T19:00:34.637 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-13T19:00:34.637 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-13T19:00:34.637 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-13T19:00:34.637 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-13T19:00:34.688 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-13T19:00:34.688 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T19:00:34.689 INFO:teuthology.orchestra.run.trial034.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T19:00:34.690 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T19:00:34.691 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: zip-3.0-35.el9.x86_64 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:34.692 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:34.726 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-base 2026-02-13T19:00:35.022 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:41 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:35.086 INFO:teuthology.orchestra.run.trial034.stdout:Package ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:00:35.103 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:35.104 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:00:35.104 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:35.124 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install cephadm 2026-02-13T19:00:35.425 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:41 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:35.508 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:35.509 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:35.509 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout: cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 1.0 M 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 1.0 M 2026-02-13T19:00:35.510 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 1.0 M 2026-02-13T19:00:35.511 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:35.635 INFO:teuthology.orchestra.run.trial034.stdout:cephadm-20.3.0-5230.gafefaa40.el9.noarch.rpm 8.1 MB/s | 1.0 MB 00:00 2026-02-13T19:00:35.635 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:35.635 INFO:teuthology.orchestra.run.trial034.stdout:Total 8.0 MB/s | 1.0 MB 00:00 2026-02-13T19:00:35.636 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:35.638 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:35.638 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:35.642 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:35.642 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:35.660 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:35.723 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:00:35.736 INFO:teuthology.orchestra.run.trial034.stdout: Installing : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:00:35.796 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:00:35.850 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-13T19:00:35.850 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:35.850 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:35.850 INFO:teuthology.orchestra.run.trial034.stdout: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:35.850 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:35.851 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:35.878 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-immutable-object-cache 2026-02-13T19:00:36.185 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:42 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:36.267 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repo Size 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 147 k 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 147 k 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 419 k 2026-02-13T19:00:36.268 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:36.354 INFO:teuthology.orchestra.run.trial034.stdout:ceph-immutable-object-cache-20.3.0-5230.gafefaa 1.7 MB/s | 147 kB 00:00 2026-02-13T19:00:36.354 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:36.355 INFO:teuthology.orchestra.run.trial034.stdout:Total 1.7 MB/s | 147 kB 00:00 2026-02-13T19:00:36.355 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:36.358 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:36.359 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:36.385 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:36.386 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:36.434 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:36.435 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:00:36.458 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:00:36.459 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:36.459 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T19:00:36.459 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:36.742 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:36.769 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-mgr 2026-02-13T19:00:37.068 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:43 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:37.134 INFO:teuthology.orchestra.run.trial034.stdout:Package ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:00:37.152 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:37.152 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:00:37.152 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:37.173 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-mgr-dashboard 2026-02-13T19:00:37.472 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:43 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:37.554 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 51 M 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T19:00:37.555 INFO:teuthology.orchestra.run.trial034.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 57 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 17 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T19:00:37.556 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:37.557 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:37.557 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:37.557 INFO:teuthology.orchestra.run.trial034.stdout:Install 13 Packages 2026-02-13T19:00:37.557 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:37.557 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 55 M 2026-02-13T19:00:37.558 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 262 M 2026-02-13T19:00:37.558 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:38.104 INFO:teuthology.orchestra.run.trial034.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5230.gafe 256 kB/s | 17 kB 00:00 2026-02-13T19:00:38.108 INFO:teuthology.orchestra.run.trial034.stdout:(2/13): ceph-grafana-dashboards-20.3.0-5230.gaf 793 kB/s | 57 kB 00:00 2026-02-13T19:00:38.585 INFO:teuthology.orchestra.run.trial034.stdout:(3/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 99 kB/s | 48 kB 00:00 2026-02-13T19:00:38.676 INFO:teuthology.orchestra.run.trial034.stdout:(4/13): protobuf-compiler-3.14.0-17.el9.x86_64. 9.3 MB/s | 862 kB 00:00 2026-02-13T19:00:38.828 INFO:teuthology.orchestra.run.trial034.stdout:(5/13): python3-protobuf-3.14.0-17.el9.noarch.r 372 kB/s | 267 kB 00:00 2026-02-13T19:00:39.068 INFO:teuthology.orchestra.run.trial034.stdout:(6/13): ceph-mgr-dashboard-20.3.0-5230.gafefaa4 49 MB/s | 51 MB 00:01 2026-02-13T19:00:39.069 INFO:teuthology.orchestra.run.trial034.stdout:(7/13): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2026-02-13T19:00:39.171 INFO:teuthology.orchestra.run.trial034.stdout:(8/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-13T19:00:39.236 INFO:teuthology.orchestra.run.trial034.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 474 kB/s | 31 kB 00:00 2026-02-13T19:00:39.317 INFO:teuthology.orchestra.run.trial034.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 2.3 MB/s | 188 kB 00:00 2026-02-13T19:00:39.381 INFO:teuthology.orchestra.run.trial034.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 345 kB/s | 22 kB 00:00 2026-02-13T19:00:39.423 INFO:teuthology.orchestra.run.trial034.stdout:(12/13): python3-grpcio-tools-1.46.7-10.el9.x86 407 kB/s | 144 kB 00:00 2026-02-13T19:00:39.523 INFO:teuthology.orchestra.run.trial034.stdout:(13/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.5 MB/s | 2.0 MB 00:00 2026-02-13T19:00:39.525 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:39.525 INFO:teuthology.orchestra.run.trial034.stdout:Total 28 MB/s | 55 MB 00:01 2026-02-13T19:00:39.551 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:39.561 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:39.561 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:39.772 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:39.772 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:39.998 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:40.003 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-13T19:00:40.015 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-13T19:00:40.017 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-13T19:00:40.044 INFO:teuthology.orchestra.run.trial034.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-13T19:00:40.088 INFO:teuthology.orchestra.run.trial034.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-13T19:00:40.097 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-13T19:00:40.125 INFO:teuthology.orchestra.run.trial034.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-13T19:00:40.135 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-13T19:00:40.142 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-13T19:00:40.145 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-13T19:00:40.154 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 11/13 2026-02-13T19:00:43.027 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 12/13 2026-02-13T19:00:43.033 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-13T19:00:43.285 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-13T19:00:43.285 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 1/13 2026-02-13T19:00:43.285 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 2/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 3/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-13T19:00:43.286 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-13T19:00:43.339 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-13T19:00:43.339 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:43.339 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T19:00:43.340 INFO:teuthology.orchestra.run.trial034.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T19:00:43.341 INFO:teuthology.orchestra.run.trial034.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T19:00:43.341 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:43.341 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:43.374 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-13T19:00:43.677 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:43.759 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:43.760 INFO:teuthology.orchestra.run.trial034.stdout:======================================================================================= 2026-02-13T19:00:43.760 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:43.760 INFO:teuthology.orchestra.run.trial034.stdout:======================================================================================= 2026-02-13T19:00:43.760 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:43.760 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 7.4 M 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T19:00:43.761 INFO:teuthology.orchestra.run.trial034.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout:======================================================================================= 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout:Install 12 Packages 2026-02-13T19:00:43.762 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:43.763 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 43 M 2026-02-13T19:00:43.763 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 234 M 2026-02-13T19:00:43.763 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:44.229 INFO:teuthology.orchestra.run.trial034.stdout:(1/12): libgfortran-11.5.0-14.el9.x86_64.rpm 11 MB/s | 794 kB 00:00 2026-02-13T19:00:44.229 INFO:teuthology.orchestra.run.trial034.stdout:(2/12): libquadmath-11.5.0-14.el9.x86_64.rpm 2.6 MB/s | 184 kB 00:00 2026-02-13T19:00:44.390 INFO:teuthology.orchestra.run.trial034.stdout:(3/12): ceph-mgr-diskprediction-local-20.3.0-52 32 MB/s | 7.4 MB 00:00 2026-02-13T19:00:44.795 INFO:teuthology.orchestra.run.trial034.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2026-02-13T19:00:44.876 INFO:teuthology.orchestra.run.trial034.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 37 kB/s | 18 kB 00:00 2026-02-13T19:00:45.288 INFO:teuthology.orchestra.run.trial034.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2026-02-13T19:00:45.374 INFO:teuthology.orchestra.run.trial034.stdout:(7/12): openblas-0.3.29-1.el9.x86_64.rpm 73 kB/s | 42 kB 00:00 2026-02-13T19:00:45.988 INFO:teuthology.orchestra.run.trial034.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 4.8 MB/s | 5.3 MB 00:01 2026-02-13T19:00:46.057 INFO:teuthology.orchestra.run.trial034.stdout:(9/12): python3-devel-3.9.25-2.el9.x86_64.rpm 318 kB/s | 244 kB 00:00 2026-02-13T19:00:46.497 INFO:teuthology.orchestra.run.trial034.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 5.5 MB/s | 6.1 MB 00:01 2026-02-13T19:00:46.816 INFO:teuthology.orchestra.run.trial034.stdout:(11/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 534 kB/s | 442 kB 00:00 2026-02-13T19:00:47.469 INFO:teuthology.orchestra.run.trial034.stdout:(12/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 14 MB/s | 19 MB 00:01 2026-02-13T19:00:47.471 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:47.471 INFO:teuthology.orchestra.run.trial034.stdout:Total 12 MB/s | 43 MB 00:03 2026-02-13T19:00:47.631 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:47.644 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:47.644 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:47.848 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:47.848 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:48.066 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:48.068 INFO:teuthology.orchestra.run.trial034.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2026-02-13T19:00:48.074 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-13T19:00:48.108 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-13T19:00:48.112 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-13T19:00:48.171 INFO:teuthology.orchestra.run.trial034.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-13T19:00:48.173 INFO:teuthology.orchestra.run.trial034.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-13T19:00:48.194 INFO:teuthology.orchestra.run.trial034.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2026-02-13T19:00:48.512 INFO:teuthology.orchestra.run.trial034.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2026-02-13T19:00:48.582 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-13T19:00:49.230 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T19:00:49.685 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-13T19:00:49.686 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-13T19:00:49.960 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-13T19:00:49.960 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 1/12 2026-02-13T19:00:49.960 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T19:00:49.961 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-13T19:00:50.016 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-13T19:00:50.016 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:50.016 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:50.016 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:50.016 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T19:00:50.017 INFO:teuthology.orchestra.run.trial034.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T19:00:50.018 INFO:teuthology.orchestra.run.trial034.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T19:00:50.018 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:50.018 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:50.056 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-mgr-rook 2026-02-13T19:00:50.352 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:50.434 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:50.435 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:50.435 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:50.435 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-rook noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 50 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T19:00:50.436 INFO:teuthology.orchestra.run.trial034.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:Install 10 Packages 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 2.0 M 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 25 M 2026-02-13T19:00:50.437 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:50.716 INFO:teuthology.orchestra.run.trial034.stdout:(1/10): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2026-02-13T19:00:50.723 INFO:teuthology.orchestra.run.trial034.stdout:(2/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2026-02-13T19:00:50.739 INFO:teuthology.orchestra.run.trial034.stdout:(3/10): python3-requests-oauthlib-1.3.0-12.el9. 2.3 MB/s | 54 kB 00:00 2026-02-13T19:00:51.024 INFO:teuthology.orchestra.run.trial034.stdout:(4/10): ceph-mgr-rook-20.3.0-5230.gafefaa40.el9 112 kB/s | 50 kB 00:00 2026-02-13T19:00:51.072 INFO:teuthology.orchestra.run.trial034.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 42 kB/s | 14 kB 00:00 2026-02-13T19:00:51.105 INFO:teuthology.orchestra.run.trial034.stdout:(6/10): python3-cachetools-4.2.4-1.el9.noarch.r 84 kB/s | 32 kB 00:00 2026-02-13T19:00:51.361 INFO:teuthology.orchestra.run.trial034.stdout:(7/10): python3-google-auth-2.45.0-1.el9.noarch 754 kB/s | 254 kB 00:00 2026-02-13T19:00:51.538 INFO:teuthology.orchestra.run.trial034.stdout:(8/10): python3-kubernetes-26.1.0-3.el9.noarch. 2.2 MB/s | 1.0 MB 00:00 2026-02-13T19:00:51.541 INFO:teuthology.orchestra.run.trial034.stdout:(9/10): python3-rsa-4.9-2.el9.noarch.rpm 136 kB/s | 59 kB 00:00 2026-02-13T19:00:51.548 INFO:teuthology.orchestra.run.trial034.stdout:(10/10): python3-websocket-client-1.2.3-2.el9.n 478 kB/s | 90 kB 00:00 2026-02-13T19:00:51.550 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:51.551 INFO:teuthology.orchestra.run.trial034.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2026-02-13T19:00:51.573 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:51.581 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:51.581 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:51.620 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:51.620 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:51.693 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:51.743 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-13T19:00:51.756 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-13T19:00:51.771 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-13T19:00:51.775 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-13T19:00:51.783 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-13T19:00:51.822 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-13T19:00:51.834 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T19:00:52.152 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-13T19:00:52.166 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-13T19:00:52.167 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-13T19:00:52.271 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 1/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-13T19:00:52.272 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-13T19:00:52.327 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-13T19:00:52.327 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T19:00:52.328 INFO:teuthology.orchestra.run.trial034.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T19:00:52.329 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:52.329 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:52.358 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-mgr-cephadm 2026-02-13T19:00:52.653 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:00:58 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:52.736 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 184 k 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T19:00:52.737 INFO:teuthology.orchestra.run.trial034.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout:Install 7 Packages 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 7.1 M 2026-02-13T19:00:52.738 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 34 M 2026-02-13T19:00:52.739 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:53.035 INFO:teuthology.orchestra.run.trial034.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5230.gafefaa40.e 2.1 MB/s | 184 kB 00:00 2026-02-13T19:00:53.081 INFO:teuthology.orchestra.run.trial034.stdout:(2/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 1.9 MB/s | 249 kB 00:00 2026-02-13T19:00:53.103 INFO:teuthology.orchestra.run.trial034.stdout:(3/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 510 kB/s | 35 kB 00:00 2026-02-13T19:00:53.212 INFO:teuthology.orchestra.run.trial034.stdout:(4/7): python3-natsort-7.1.1-5.el9.noarch.rpm 530 kB/s | 58 kB 00:00 2026-02-13T19:00:53.254 INFO:teuthology.orchestra.run.trial034.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 2.0 MB/s | 86 kB 00:00 2026-02-13T19:00:53.259 INFO:teuthology.orchestra.run.trial034.stdout:(6/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.0 MB/s | 548 kB 00:00 2026-02-13T19:00:53.822 INFO:teuthology.orchestra.run.trial034.stdout:(7/7): python3-babel-2.9.1-2.el9.noarch.rpm 6.8 MB/s | 6.0 MB 00:00 2026-02-13T19:00:53.823 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:53.824 INFO:teuthology.orchestra.run.trial034.stdout:Total 6.5 MB/s | 7.1 MB 00:01 2026-02-13T19:00:53.861 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:53.868 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:53.868 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:53.939 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:53.939 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:54.028 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:54.077 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-13T19:00:54.087 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-13T19:00:54.092 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-13T19:00:54.313 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-13T19:00:54.336 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-13T19:00:54.361 INFO:teuthology.orchestra.run.trial034.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-13T19:00:54.362 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-13T19:00:54.470 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-13T19:00:54.470 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/7 2026-02-13T19:00:54.471 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-13T19:00:54.471 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-13T19:00:54.471 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-13T19:00:54.471 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-13T19:00:54.471 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T19:00:54.527 INFO:teuthology.orchestra.run.trial034.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T19:00:54.528 INFO:teuthology.orchestra.run.trial034.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T19:00:54.528 INFO:teuthology.orchestra.run.trial034.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T19:00:54.528 INFO:teuthology.orchestra.run.trial034.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T19:00:54.528 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:54.528 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:54.558 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-osd 2026-02-13T19:00:54.862 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:54.928 INFO:teuthology.orchestra.run.trial034.stdout:Package ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:00:54.946 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:54.947 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:00:54.947 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:54.968 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-osd-classic 2026-02-13T19:00:55.269 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:55.333 INFO:teuthology.orchestra.run.trial034.stdout:Package ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:00:55.351 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:55.352 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:00:55.352 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:55.372 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-fuse 2026-02-13T19:00:55.677 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:55.760 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:55.761 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:55.761 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repo Size 2026-02-13T19:00:55.761 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout: ceph-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 987 k 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 987 k 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 2.9 M 2026-02-13T19:00:55.762 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:55.890 INFO:teuthology.orchestra.run.trial034.stdout:ceph-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 7.5 MB/s | 987 kB 00:00 2026-02-13T19:00:55.891 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:55.891 INFO:teuthology.orchestra.run.trial034.stdout:Total 7.5 MB/s | 987 kB 00:00 2026-02-13T19:00:55.891 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:55.895 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:55.895 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:55.931 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:55.931 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:56.023 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:56.040 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:56.253 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:56.308 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:56.309 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:56.309 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:56.309 INFO:teuthology.orchestra.run.trial034.stdout: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:56.309 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:56.309 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:56.336 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install ceph-volume 2026-02-13T19:00:56.639 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:56.724 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:56.724 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repository Size 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout: ceph-volume noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 299 k 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:56.725 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:56.726 INFO:teuthology.orchestra.run.trial034.stdout:Install 2 Packages 2026-02-13T19:00:56.726 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:56.726 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 649 k 2026-02-13T19:00:56.726 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 2.2 M 2026-02-13T19:00:56.726 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:57.190 INFO:teuthology.orchestra.run.trial034.stdout:(1/2): ceph-volume-20.3.0-5230.gafefaa40.el9.no 3.1 MB/s | 299 kB 00:00 2026-02-13T19:00:57.673 INFO:teuthology.orchestra.run.trial034.stdout:(2/2): cryptsetup-2.8.1-2.el9.x86_64.rpm 607 kB/s | 350 kB 00:00 2026-02-13T19:00:57.674 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:57.674 INFO:teuthology.orchestra.run.trial034.stdout:Total 684 kB/s | 649 kB 00:00 2026-02-13T19:00:57.684 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:57.689 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:57.689 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:57.733 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:57.733 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:57.812 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:57.875 INFO:teuthology.orchestra.run.trial034.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 1/2 2026-02-13T19:00:57.877 INFO:teuthology.orchestra.run.trial034.stdout: Installing : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-13T19:00:57.891 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-13T19:00:57.891 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:00:57.891 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T19:00:57.892 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.107 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 1/2 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 2/2 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.164 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:58.191 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install librados-devel 2026-02-13T19:00:58.492 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:58.576 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repo Size 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout: librados-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 123 k 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.577 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 123 k 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 437 k 2026-02-13T19:00:58.578 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:58.662 INFO:teuthology.orchestra.run.trial034.stdout:librados-devel-20.3.0-5230.gafefaa40.el9.x86_64 1.4 MB/s | 123 kB 00:00 2026-02-13T19:00:58.662 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:58.662 INFO:teuthology.orchestra.run.trial034.stdout:Total 1.4 MB/s | 123 kB 00:00 2026-02-13T19:00:58.663 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:58.665 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:58.665 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:58.698 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:58.698 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:00:58.752 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:00:58.769 INFO:teuthology.orchestra.run.trial034.stdout: Installing : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:58.846 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:58.902 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:00:58.902 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.902 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:00:58.902 INFO:teuthology.orchestra.run.trial034.stdout: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:00:58.902 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:58.903 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:58.931 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install libcephfs2 2026-02-13T19:00:59.233 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:59.298 INFO:teuthology.orchestra.run.trial034.stdout:Package libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:00:59.316 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:59.317 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:00:59.317 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:00:59.338 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install libcephfs-devel 2026-02-13T19:00:59.637 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:00:59.720 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repo Size 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 37 k 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout:Installing dependencies: 2026-02-13T19:00:59.721 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:Install 2 Packages 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 62 k 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 223 k 2026-02-13T19:00:59.722 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:00:59.962 INFO:teuthology.orchestra.run.trial034.stdout:(1/2): libcephfs-devel-20.3.0-5230.gafefaa40.el 154 kB/s | 37 kB 00:00 2026-02-13T19:00:59.963 INFO:teuthology.orchestra.run.trial034.stdout:(2/2): libcephfs-proxy2-20.3.0-5230.gafefaa40.e 102 kB/s | 25 kB 00:00 2026-02-13T19:00:59.963 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:00:59.963 INFO:teuthology.orchestra.run.trial034.stdout:Total 255 kB/s | 62 kB 00:00 2026-02-13T19:00:59.963 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:00:59.966 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:00:59.966 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:00:59.999 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:00:59.999 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:01:00.047 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:01:00.048 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:01:00.063 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:01:00.079 INFO:teuthology.orchestra.run.trial034.stdout: Installing : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:01:00.130 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:01:00.130 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-13T19:01:00.185 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-13T19:01:00.185 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:00.185 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:01:00.186 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:01:00.186 INFO:teuthology.orchestra.run.trial034.stdout: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:01:00.186 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:00.186 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:00.211 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install librados2 2026-02-13T19:01:00.514 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:00.578 INFO:teuthology.orchestra.run.trial034.stdout:Package librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:00.597 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:00.597 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:00.597 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:00.618 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install librbd1 2026-02-13T19:01:00.921 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:00.985 INFO:teuthology.orchestra.run.trial034.stdout:Package librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:01.004 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:01.004 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:01.004 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:01.024 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install python3-rados 2026-02-13T19:01:01.319 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:01.384 INFO:teuthology.orchestra.run.trial034.stdout:Package python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:01.402 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:01.402 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:01.403 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:01.424 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install python3-rgw 2026-02-13T19:01:01.721 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:01.786 INFO:teuthology.orchestra.run.trial034.stdout:Package python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:01.804 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:01.805 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:01.805 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:01.826 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install python3-cephfs 2026-02-13T19:01:02.128 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:02.193 INFO:teuthology.orchestra.run.trial034.stdout:Package python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:02.212 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:02.212 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:02.212 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:02.233 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install python3-rbd 2026-02-13T19:01:02.533 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:02.597 INFO:teuthology.orchestra.run.trial034.stdout:Package python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-13T19:01:02.615 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:02.616 INFO:teuthology.orchestra.run.trial034.stdout:Nothing to do. 2026-02-13T19:01:02.616 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:02.638 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install rbd-fuse 2026-02-13T19:01:02.938 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:03.022 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout: Package Architecture Version Repository Size 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout: rbd-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 84 k 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:01:03.023 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.024 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:01:03.024 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.024 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 84 k 2026-02-13T19:01:03.024 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 209 k 2026-02-13T19:01:03.024 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:01:03.096 INFO:teuthology.orchestra.run.trial034.stdout:rbd-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2026-02-13T19:01:03.097 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:01:03.097 INFO:teuthology.orchestra.run.trial034.stdout:Total 1.1 MB/s | 84 kB 00:00 2026-02-13T19:01:03.097 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:01:03.101 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:01:03.101 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:01:03.132 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:01:03.133 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:01:03.183 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:01:03.199 INFO:teuthology.orchestra.run.trial034.stdout: Installing : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:03.277 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.334 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:03.363 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install rbd-mirror 2026-02-13T19:01:03.663 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:09 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:03.746 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:03.747 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.747 INFO:teuthology.orchestra.run.trial034.stdout: Package Arch Version Repo Size 2026-02-13T19:01:03.747 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.747 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout: rbd-mirror x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.9 M 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 2.9 M 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 11 M 2026-02-13T19:01:03.748 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:01:03.892 INFO:teuthology.orchestra.run.trial034.stdout:rbd-mirror-20.3.0-5230.gafefaa40.el9.x86_64.rpm 20 MB/s | 2.9 MB 00:00 2026-02-13T19:01:03.893 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:01:03.893 INFO:teuthology.orchestra.run.trial034.stdout:Total 20 MB/s | 2.9 MB 00:00 2026-02-13T19:01:03.893 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:01:03.898 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:01:03.898 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:01:03.937 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:01:03.938 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:01:04.121 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:01:04.123 INFO:teuthology.orchestra.run.trial034.stdout: Installing : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:04.145 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:04.146 INFO:teuthology.orchestra.run.trial034.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T19:01:04.146 INFO:teuthology.orchestra.run.trial034.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T19:01:04.146 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T19:01:04.146 INFO:teuthology.orchestra.run.trial034.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T19:01:04.146 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:04.464 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:04.465 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:04.465 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:01:04.465 INFO:teuthology.orchestra.run.trial034.stdout: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:01:04.465 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:04.465 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:04.492 DEBUG:teuthology.orchestra.run.trial034:> sudo yum -y install rbd-nbd 2026-02-13T19:01:04.792 INFO:teuthology.orchestra.run.trial034.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Feb 2026 06:59:54 PM UTC. 2026-02-13T19:01:04.875 INFO:teuthology.orchestra.run.trial034.stdout:Dependencies resolved. 2026-02-13T19:01:04.875 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout: Package Architecture Version Repository Size 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout:Installing: 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout: rbd-nbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 172 k 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout:Transaction Summary 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout:================================================================================ 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout:Install 1 Package 2026-02-13T19:01:04.876 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:04.877 INFO:teuthology.orchestra.run.trial034.stdout:Total download size: 172 k 2026-02-13T19:01:04.877 INFO:teuthology.orchestra.run.trial034.stdout:Installed size: 462 k 2026-02-13T19:01:04.877 INFO:teuthology.orchestra.run.trial034.stdout:Downloading Packages: 2026-02-13T19:01:04.961 INFO:teuthology.orchestra.run.trial034.stdout:rbd-nbd-20.3.0-5230.gafefaa40.el9.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2026-02-13T19:01:04.962 INFO:teuthology.orchestra.run.trial034.stdout:-------------------------------------------------------------------------------- 2026-02-13T19:01:04.962 INFO:teuthology.orchestra.run.trial034.stdout:Total 1.9 MB/s | 172 kB 00:00 2026-02-13T19:01:04.963 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction check 2026-02-13T19:01:04.967 INFO:teuthology.orchestra.run.trial034.stdout:Transaction check succeeded. 2026-02-13T19:01:04.967 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction test 2026-02-13T19:01:04.999 INFO:teuthology.orchestra.run.trial034.stdout:Transaction test succeeded. 2026-02-13T19:01:04.999 INFO:teuthology.orchestra.run.trial034.stdout:Running transaction 2026-02-13T19:01:05.054 INFO:teuthology.orchestra.run.trial034.stdout: Preparing : 1/1 2026-02-13T19:01:05.071 INFO:teuthology.orchestra.run.trial034.stdout: Installing : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:05.147 INFO:teuthology.orchestra.run.trial034.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:05.204 INFO:teuthology.orchestra.run.trial034.stdout: Verifying : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-13T19:01:05.204 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:05.204 INFO:teuthology.orchestra.run.trial034.stdout:Installed: 2026-02-13T19:01:05.204 INFO:teuthology.orchestra.run.trial034.stdout: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-13T19:01:05.205 INFO:teuthology.orchestra.run.trial034.stdout: 2026-02-13T19:01:05.205 INFO:teuthology.orchestra.run.trial034.stdout:Complete! 2026-02-13T19:01:05.235 DEBUG:teuthology.parallel:result is None 2026-02-13T19:01:05.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:01:05.405 DEBUG:teuthology.orchestra.run.trial034:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T19:01:05.426 INFO:teuthology.orchestra.run.trial034.stdout:20.3.0-5230.gafefaa40.el9 2026-02-13T19:01:05.427 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5230.gafefaa40.el9 2026-02-13T19:01:05.427 INFO:teuthology.task.install:The correct ceph version 20.3.0-5230.gafefaa40 is installed. 2026-02-13T19:01:05.429 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-13T19:01:05.430 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T19:01:05.430 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T19:01:05.494 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-13T19:01:05.495 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T19:01:05.495 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T19:01:05.558 DEBUG:teuthology.orchestra.run.trial034:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T19:01:05.621 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-13T19:01:05.621 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T19:01:05.622 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T19:01:05.684 DEBUG:teuthology.orchestra.run.trial034:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T19:01:05.748 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-13T19:01:05.748 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T19:01:05.748 DEBUG:teuthology.orchestra.run.trial034:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T19:01:05.813 DEBUG:teuthology.orchestra.run.trial034:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T19:01:05.876 INFO:teuthology.run_tasks:Running task workunit... 2026-02-13T19:01:05.901 INFO:tasks.workunit:Pulling workunits from ref afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:01:05.901 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-13T19:01:05.901 INFO:tasks.workunit:timeout=3h 2026-02-13T19:01:05.902 INFO:tasks.workunit:cleanup=True 2026-02-13T19:01:05.902 DEBUG:teuthology.orchestra.run.trial034:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-13T19:01:05.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:01:05.933 INFO:teuthology.orchestra.run.trial034.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-13T19:01:05.933 DEBUG:teuthology.orchestra.run.trial034:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-13T19:01:05.989 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-13T19:01:05.990 DEBUG:teuthology.orchestra.run.trial034:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-13T19:01:06.046 DEBUG:teuthology.orchestra.run.trial034:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-13T19:01:06.109 INFO:tasks.workunit.client.0.trial034.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:Note: switching to 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'. 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:state without impacting any branches by switching back to a branch. 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: git switch -c 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:Or undo this operation with: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: git switch - 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr: 2026-02-13T19:01:25.869 INFO:tasks.workunit.client.0.trial034.stderr:HEAD is now at afefaa40503 Merge branch 'block_non_4k_chunks' of https://github.com/aainscow/ceph into wip-yuri4-testing-2026-02-11-2126 2026-02-13T19:01:25.875 DEBUG:teuthology.orchestra.run.trial034:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-13T19:01:25.933 DEBUG:teuthology.orchestra.run.trial034:> set -ex 2026-02-13T19:01:25.934 DEBUG:teuthology.orchestra.run.trial034:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-13T19:01:25.990 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-13T19:01:25.991 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-13T19:01:25.991 DEBUG:teuthology.orchestra.run.trial034:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ TIMEOUT=300 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ MAX_TIMEOUT=15 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ PG_NUM=4 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ TMPDIR=/tmp 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:+++ uname 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ '[' Linux = FreeBSD ']' 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ SED=sed 2026-02-13T19:01:26.052 INFO:tasks.workunit.client.0.trial034.stderr:++ AWK=awk 2026-02-13T19:01:26.053 INFO:tasks.workunit.client.0.trial034.stderr:+++ stty -a 2026-02-13T19:01:26.053 INFO:tasks.workunit.client.0.trial034.stderr:+++ head -1 2026-02-13T19:01:26.053 INFO:tasks.workunit.client.0.trial034.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-13T19:01:26.053 INFO:tasks.workunit.client.0.trial034.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:01:26.054 INFO:tasks.workunit.client.0.trial034.stderr:++ termwidth= 2026-02-13T19:01:26.054 INFO:tasks.workunit.client.0.trial034.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-13T19:01:26.054 INFO:tasks.workunit.client.0.trial034.stderr:++ DIFFCOLOPTS='-y ' 2026-02-13T19:01:26.054 INFO:tasks.workunit.client.0.trial034.stderr:++ KERNCORE=kernel.core_pattern 2026-02-13T19:01:26.054 INFO:tasks.workunit.client.0.trial034.stderr:++ EXTRA_OPTS= 2026-02-13T19:01:26.055 INFO:tasks.workunit.client.0.trial034.stderr:++ test '' = TESTS 2026-02-13T19:01:26.056 INFO:tasks.workunit.client.0.trial034.stderr:++ uname -m 2026-02-13T19:01:26.057 INFO:tasks.workunit.client.0.trial034.stderr:+ arch=x86_64 2026-02-13T19:01:26.057 INFO:tasks.workunit.client.0.trial034.stderr:+ case $arch in 2026-02-13T19:01:26.057 INFO:tasks.workunit.client.0.trial034.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-13T19:01:26.057 INFO:tasks.workunit.client.0.trial034.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-13T19:01:26.057 INFO:tasks.workunit.client.0.trial034.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:+ main test-erasure-code-plugins 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:+ shift 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:+ shopt -s -o xtrace 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code-plugins 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-13T19:01:26.058 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-13T19:01:26.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none ' 2026-02-13T19:01:26.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:26.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-13T19:01:26.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_profile_warning 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:TEST_preload_no_warning 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:TEST_preload_no_warning_default 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:TEST_preload_warning' 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:01:26.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:01:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:01:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:01:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:01:26.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:01:26.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:01:26.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:01:26.064 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:01:26.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:01:26.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:01:26.065 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:01:26.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:26.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:01:26.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:01:26.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:26.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:01:26.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:01:26.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:01:26.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:01:26.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:01:26.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:01:26.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:01:26.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:01:26.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:01:26.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:01:26.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:01:26.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:01:26.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:01:26.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:01:26.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:26.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:01:26.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:01:26.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:01:26.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:01:26.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:01:26.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:01:26.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.118 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.119 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:01:26.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:01:26.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:01:26.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:01:26.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:01:26.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:01:26.162 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:01:26.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:01:26.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:01:26.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:01:26.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:01:26.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:01:26.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.299 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:26.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:01:26.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:01:26.315 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:26.316 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:26.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:01:26.318 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9b9cd633-d34b-4c10-b91b-c62fb4cad7fd 2026-02-13T19:01:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9b9cd633-d34b-4c10-b91b-c62fb4cad7fd' 2026-02-13T19:01:26.318 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 9b9cd633-d34b-4c10-b91b-c62fb4cad7fd 2026-02-13T19:01:26.318 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:26.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAGdY9pjh6EExAAhlGBhbYwQsXKpJyw8q/A0g== 2026-02-13T19:01:26.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAGdY9pjh6EExAAhlGBhbYwQsXKpJyw8q/A0g=="}' 2026-02-13T19:01:26.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9b9cd633-d34b-4c10-b91b-c62fb4cad7fd -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:01:26.427 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:26.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:01:26.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAGdY9pjh6EExAAhlGBhbYwQsXKpJyw8q/A0g== --osd-uuid 9b9cd633-d34b-4c10-b91b-c62fb4cad7fd 2026-02-13T19:01:26.448 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:26.447+0000 7f5dff82c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:26.450 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:26.448+0000 7f5dff82c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:26.450 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:26.449+0000 7f5dff82c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:26.451 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:26.449+0000 7f5dff82c780 -1 bdev(0x24b6c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:26.451 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:26.449+0000 7f5dff82c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:01:26.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:01:26.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:26.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:01:26.884 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:01:26.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:26.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:01:26.988 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:01:26.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:26.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:26.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:26.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:27.002 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:27.001+0000 7f3c68247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:27.004 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:27.002+0000 7f3c68247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:27.005 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:27.003+0000 7f3c68247780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:27.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:01:27.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:27.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:01:27.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:27.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:27.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:27.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:27.086 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:27.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:27.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:01:27.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:27.202 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:27.201+0000 7f3c68247780 -1 Falling back to public interface 2026-02-13T19:01:27.600 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:27.598+0000 7f3c68247780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:01:28.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:28.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:28.181 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:28.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:28.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:28.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:01:28.274 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2084308045,v1:127.0.0.1:6801/2084308045] [v2:127.0.0.1:6802/2084308045,v1:127.0.0.1:6803/2084308045] exists,up 9b9cd633-d34b-4c10-b91b-c62fb4cad7fd 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:28.276 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:28.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-13T19:01:28.278 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:28.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5cd8f715-e99f-44f4-a143-e6ca46b4afb2 2026-02-13T19:01:28.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5cd8f715-e99f-44f4-a143-e6ca46b4afb2' 2026-02-13T19:01:28.278 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 5cd8f715-e99f-44f4-a143-e6ca46b4afb2 2026-02-13T19:01:28.278 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:28.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAIdY9pEgsWERAAlRLBBxqPU9suh4s5fBJXxQ== 2026-02-13T19:01:28.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAIdY9pEgsWERAAlRLBBxqPU9suh4s5fBJXxQ=="}' 2026-02-13T19:01:28.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5cd8f715-e99f-44f4-a143-e6ca46b4afb2 -i td/test-erasure-code-plugins/1/new.json 2026-02-13T19:01:28.386 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:28.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-13T19:01:28.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAIdY9pEgsWERAAlRLBBxqPU9suh4s5fBJXxQ== --osd-uuid 5cd8f715-e99f-44f4-a143-e6ca46b4afb2 2026-02-13T19:01:28.407 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:28.405+0000 7f779a837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:28.409 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:28.407+0000 7f779a837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:28.409 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:28.408+0000 7f779a837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:28.410 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:28.408+0000 7f779a837780 -1 bdev(0x3506c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:28.410 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:28.408+0000 7f779a837780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-13T19:01:28.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-13T19:01:28.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:28.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:01:28.798 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:01:28.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:29.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:01:29.119 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:01:29.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:29.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:29.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:29.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:29.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:29.130+0000 7fa904fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:29.134 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:29.132+0000 7fa904fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:29.135 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:29.133+0000 7fa904fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:29.271 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:29.268+0000 7fa904fd8780 -1 Falling back to public interface 2026-02-13T19:01:29.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:01:29.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:29.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:01:29.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:29.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:29.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:29.343 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:29.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:29.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:29.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:01:29.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:29.597 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:29.595+0000 7fa904fd8780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:01:30.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:30.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:30.567 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:30.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:30.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:30.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:01:30.794 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1122777504,v1:127.0.0.1:6811/1122777504] [v2:127.0.0.1:6812/1122777504,v1:127.0.0.1:6813/1122777504] exists,up 5cd8f715-e99f-44f4-a143-e6ca46b4afb2 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:01:30.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:30.799 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:30.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=61804500-52a9-414f-9a7c-c849489ddf66 2026-02-13T19:01:30.801 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 61804500-52a9-414f-9a7c-c849489ddf66 2026-02-13T19:01:30.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 61804500-52a9-414f-9a7c-c849489ddf66' 2026-02-13T19:01:30.802 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:30.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAKdY9pdZ8CMBAArwHTS0LmN5MmAr25sWupmw== 2026-02-13T19:01:30.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAKdY9pdZ8CMBAArwHTS0LmN5MmAr25sWupmw=="}' 2026-02-13T19:01:30.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 61804500-52a9-414f-9a7c-c849489ddf66 -i td/test-erasure-code-plugins/2/new.json 2026-02-13T19:01:31.033 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:01:31.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-13T19:01:31.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAKdY9pdZ8CMBAArwHTS0LmN5MmAr25sWupmw== --osd-uuid 61804500-52a9-414f-9a7c-c849489ddf66 2026-02-13T19:01:31.056 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.054+0000 7f7f33837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.057 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.055+0000 7f7f33837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.058 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.056+0000 7f7f33837780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.058 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.056+0000 7f7f33837780 -1 bdev(0x30cec00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:31.058 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.056+0000 7f7f33837780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-13T19:01:31.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-13T19:01:31.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:31.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:01:31.371 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:01:31.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:31.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:01:31.685 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:01:31.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:31.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:31.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:31.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:31.698 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.696+0000 7fde5e231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.699 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.698+0000 7fde5e231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.700 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.698+0000 7fde5e231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:31.828 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:31.826+0000 7fde5e231780 -1 Falling back to public interface 2026-02-13T19:01:31.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:01:31.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:31.920 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:31.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:31.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:01:32.114 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:32.112+0000 7fde5e231780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:01:32.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:33.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:33.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:33.151 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:33.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:33.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:33.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:01:33.378 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3151413009,v1:127.0.0.1:6819/3151413009] [v2:127.0.0.1:6820/3151413009,v1:127.0.0.1:6821/3151413009] exists,up 61804500-52a9-414f-9a7c-c849489ddf66 2026-02-13T19:01:33.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:33.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:33.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:33.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-13T19:01:33.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:01:33.594 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' does not exist 2026-02-13T19:01:33.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:01:33.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:01:33.878 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:01:33.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:01:34.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:01:35.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-13T19:01:35.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:01:35.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:01:35.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:01:35.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:01:35.167 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:01:35.168 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:01:35.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:01:35.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:01:35.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:01:35.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:01:35.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:01:35.473 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:01:35.473 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:01:35.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:01:35.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:35.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:01:35.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T19:01:35.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T19:01:35.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T19:01:35.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:35.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:01:35.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-13T19:01:35.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-13T19:01:35.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-13T19:01:35.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:35.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:01:35.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-13T19:01:35.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-13T19:01:35.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963 2-64424509443' 2026-02-13T19:01:35.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:01:35.662 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-13T19:01:35.662 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:01:35.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:01:35.664 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T19:01:35.664 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:01:35.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-13T19:01:35.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T19:01:35.665 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836484 2026-02-13T19:01:35.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:01:35.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T19:01:35.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:01:36.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:01:36.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:01:37.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T19:01:37.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:01:37.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-13T19:01:37.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:01:37.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:01:37.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-13T19:01:37.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:01:37.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-13T19:01:37.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-13T19:01:37.119 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949672963 2026-02-13T19:01:37.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:01:37.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672963 2026-02-13T19:01:37.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:01:37.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509443 2026-02-13T19:01:37.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:01:37.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:01:37.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-13T19:01:37.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:01:37.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-13T19:01:37.347 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 64424509443 2026-02-13T19:01:37.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-13T19:01:37.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:01:37.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-13T19:01:37.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:01:37.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:01:37.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:01:37.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:01:37.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:01:38.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:01:38.114 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:01:38.114 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:01:38.114 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:01:38.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:01:38.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:01:38.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:01:38.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-13T19:01:38.424 INFO:tasks.workunit.client.0.trial034.stdout:Starting test for blaum-roth profile health warning 2026-02-13T19:01:38.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-13T19:01:38.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-13T19:01:39.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-13T19:01:39.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:01:39.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:01:39.018 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:01:39.019 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:39.019 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:39.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:01:39.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-13T19:01:39.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:01:39.056 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:01:39.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-13T19:01:49.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:01:49.066 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:01:41.114+0000 7fd850ff6640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-13T19:01:49.066 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:01:41.114+0000 7fd84e7f1640 10 mon.a@0(leader).log v75 logging 2026-02-13T19:01:41.115354+0000 mon.a (mon.0) 144 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-13T19:01:49.066 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:01:41.165+0000 7fd84bfec640 7 mon.a@0(leader).log v76 update_from_paxos applying incremental log 76 2026-02-13T19:01:41.115354+0000 mon.a (mon.0) 144 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.067 INFO:tasks.workunit.client.0.trial034.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-13T19:01:49.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:01:49.068 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.069 INFO:tasks.workunit.client.0.trial034.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-13T19:01:49.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:01:49.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:01:49.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:01:49.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:01:49.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:01:49.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:01:49.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:01:49.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:01:49.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:01:49.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:49.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:01:49.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:01:49.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:49.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:01:49.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:01:49.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:01:49.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:01:49.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:01:49.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:01:49.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:01:49.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-13T19:01:49.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-13T19:01:49.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:01:49.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:01:49.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:01:49.195 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:01:49.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:01:49.195 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:01:49.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:01:49.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:01:49.196 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:01:49.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:49.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:01:49.198 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:01:49.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:01:49.198 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:01:49.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:01:49.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:01:49.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:01:49.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:01:49.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:01:49.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:01:49.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:01:49.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:01:49.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:01:49.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:01:49.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:01:49.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:01:49.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:01:49.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:49.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:01:49.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:01:49.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:01:49.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:01:49.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:01:49.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:01:49.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:01:49.289 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:01:49.289 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:01:49.289 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:01:49.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:01:49.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:01:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:01:49.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:01:49.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:49.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:49.429 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:49.429 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.429 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:49.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:01:49.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:01:49.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-13T19:01:49.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-13T19:01:49.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-13T19:01:49.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:49.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:49.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:49.448 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:49.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=35cfec10-9bbe-429a-8db2-392a0cfef243 2026-02-13T19:01:49.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 35cfec10-9bbe-429a-8db2-392a0cfef243' 2026-02-13T19:01:49.451 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 35cfec10-9bbe-429a-8db2-392a0cfef243 2026-02-13T19:01:49.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:49.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAddY9po50pGxAAREdBOdcIdgqKY/RITOdVMw== 2026-02-13T19:01:49.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAddY9po50pGxAAREdBOdcIdgqKY/RITOdVMw=="}' 2026-02-13T19:01:49.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 35cfec10-9bbe-429a-8db2-392a0cfef243 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:01:49.557 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:49.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:01:49.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAddY9po50pGxAAREdBOdcIdgqKY/RITOdVMw== --osd-uuid 35cfec10-9bbe-429a-8db2-392a0cfef243 2026-02-13T19:01:49.578 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:49.578+0000 7f2115062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:49.580 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:49.579+0000 7f2115062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:49.581 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:49.580+0000 7f2115062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:49.581 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:49.580+0000 7f2115062780 -1 bdev(0x40f0c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:49.581 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:49.580+0000 7f2115062780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:01:49.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:01:49.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:49.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:01:49.972 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:01:49.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:50.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:01:50.075 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:01:50.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:50.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:50.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:50.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:50.088 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:50.088+0000 7efde9012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:50.090 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:50.089+0000 7efde9012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:50.091 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:50.090+0000 7efde9012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:50.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:01:50.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:50.317 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:50.317+0000 7efde9012780 -1 Falling back to public interface 2026-02-13T19:01:50.675 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:50.674+0000 7efde9012780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:01:51.366 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/110089789,v1:127.0.0.1:6801/110089789] [v2:127.0.0.1:6802/110089789,v1:127.0.0.1:6803/110089789] exists,up 35cfec10-9bbe-429a-8db2-392a0cfef243 2026-02-13T19:01:51.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:51.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:51.372 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:51.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3' 2026-02-13T19:01:51.374 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3 2026-02-13T19:01:51.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:51.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAfdY9pWxCKFhAAW4Y1+5ziTUZSQjnM6209wQ== 2026-02-13T19:01:51.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAfdY9pWxCKFhAAW4Y1+5ziTUZSQjnM6209wQ=="}' 2026-02-13T19:01:51.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3 -i td/test-erasure-code-plugins/1/new.json 2026-02-13T19:01:51.480 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:51.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-13T19:01:51.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAfdY9pWxCKFhAAW4Y1+5ziTUZSQjnM6209wQ== --osd-uuid d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3 2026-02-13T19:01:51.502 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:51.501+0000 7f3a57267780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:51.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:51.502+0000 7f3a57267780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:51.504 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:51.503+0000 7f3a57267780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:51.504 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:51.503+0000 7f3a57267780 -1 bdev(0x2502c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:51.505 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:51.503+0000 7f3a57267780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-13T19:01:51.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-13T19:01:51.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:51.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:01:51.809 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:01:51.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:52.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:01:52.127 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:01:52.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:52.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:52.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:52.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:52.140 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:52.139+0000 7f1141e1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:52.141 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:52.140+0000 7f1141e1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:52.142 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:52.141+0000 7f1141e1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:52.282 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:52.281+0000 7f1141e1b780 -1 Falling back to public interface 2026-02-13T19:01:52.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:52.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:52.352 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:52.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:52.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:01:52.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:52.638 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:52.637+0000 7f1141e1b780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:01:53.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:53.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:53.586 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:53.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:53.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:53.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:01:53.812 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1728594430,v1:127.0.0.1:6811/1728594430] [v2:127.0.0.1:6812/1728594430,v1:127.0.0.1:6813/1728594430] exists,up d1c29d3c-4c40-4a0a-bd2a-2178aab0a0d3 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-13T19:01:53.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:01:53.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2c0b1b0d-4ee2-4218-9dec-b746987ce015 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2c0b1b0d-4ee2-4218-9dec-b746987ce015' 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 2c0b1b0d-4ee2-4218-9dec-b746987ce015 2026-02-13T19:01:53.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:01:53.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAhdY9pkOkiMRAADQGFZ6Qi8c+ZSgTkUqiJrA== 2026-02-13T19:01:53.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAhdY9pkOkiMRAADQGFZ6Qi8c+ZSgTkUqiJrA=="}' 2026-02-13T19:01:53.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2c0b1b0d-4ee2-4218-9dec-b746987ce015 -i td/test-erasure-code-plugins/2/new.json 2026-02-13T19:01:54.049 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:01:54.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-13T19:01:54.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAhdY9pkOkiMRAADQGFZ6Qi8c+ZSgTkUqiJrA== --osd-uuid 2c0b1b0d-4ee2-4218-9dec-b746987ce015 2026-02-13T19:01:54.072 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.071+0000 7f1fefffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.073 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.072+0000 7f1fefffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.074 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.073+0000 7f1fefffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.075 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.073+0000 7f1fefffd780 -1 bdev(0x36e0c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:01:54.075 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.073+0000 7f1fefffd780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-13T19:01:54.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-13T19:01:54.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:01:54.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:01:54.398 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:01:54.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:01:54.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:01:54.709 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:01:54.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:01:54.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:01:54.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:01:54.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:01:54.723 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.722+0000 7fdf6143c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.725 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.724+0000 7fdf6143c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.726 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.725+0000 7fdf6143c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:01:54.843 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:54.842+0000 7fdf6143c780 -1 Falling back to public interface 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:01:54.932 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:01:54.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:54.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:01:55.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:01:55.194 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:01:55.193+0000 7fdf6143c780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:01:56.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:01:56.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:01:56.157 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:01:56.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:01:56.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:01:56.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:01:56.384 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4042516768,v1:127.0.0.1:6819/4042516768] [v2:127.0.0.1:6820/4042516768,v1:127.0.0.1:6821/4042516768] exists,up 2c0b1b0d-4ee2-4218-9dec-b746987ce015 2026-02-13T19:01:56.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:01:56.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:01:56.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:01:56.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-13T19:01:56.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:01:56.643 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' does not exist 2026-02-13T19:01:56.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:01:56.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:01:56.925 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:01:56.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:01:57.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:01:58.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:01:58.216 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:01:58.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:01:58.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:01:58.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:01:58.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:01:58.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:01:58.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:58.533 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:01:58.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-13T19:01:58.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-13T19:01:58.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-13T19:01:58.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:58.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:01:58.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-13T19:01:58.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-13T19:01:58.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-13T19:01:58.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:01:58.657 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542147' 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-13T19:01:58.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:01:58.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:01:58.721 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-13T19:01:58.721 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:01:58.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-13T19:01:58.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-13T19:01:58.723 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836484 2026-02-13T19:01:58.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:01:58.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-13T19:01:58.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:01:59.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:01:59.944 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:02:00.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-13T19:02:00.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:02:00.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-13T19:02:00.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:02:00.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:02:00.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-13T19:02:00.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:02:00.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-13T19:02:00.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-13T19:02:00.170 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 38654705667 2026-02-13T19:02:00.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:02:00.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-13T19:02:00.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:02:00.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542147 2026-02-13T19:02:00.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:02:00.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:02:00.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-13T19:02:00.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:02:00.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542147 2026-02-13T19:02:00.401 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542147 2026-02-13T19:02:00.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-13T19:02:00.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:02:00.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-13T19:02:00.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:02:00.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:02:00.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:02:00.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:02:00.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:02:01.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:02:01.165 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:02:01.165 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:02:01.165 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:02:01.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:02:01.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:02:01.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:02:01.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-13T19:02:01.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-13T19:02:01.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:01.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:01.801 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:01.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:01.809 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:01.636+0000 7fefc01f1640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:01.809 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:01.752+0000 7fefc01f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:01.809 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:01.752+0000 7fefc01f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:01.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-13T19:02:01.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-13T19:02:02.104 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:02.104 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:02.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:02.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:02.142 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:02.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:02.148 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:01.974+0000 7fefc01f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.148 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.093+0000 7fefc01f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.148 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.093+0000 7fefc01f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-13T19:02:02.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-13T19:02:02.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:02.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:02.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:02.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:02.473 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:02.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:02.479 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.307+0000 7fefc01f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.479 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.425+0000 7fefc01f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.479 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.425+0000 7fefc01f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-13T19:02:02.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-13T19:02:02.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-13T19:02:02.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:02.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:02.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:02.767 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:02.767 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:02.768 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:02.768 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:02.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:02.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:02.805 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:02.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:02.811 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.641+0000 7fefc01f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-13T19:02:02.811 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.757+0000 7fefc01f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-13T19:02:02.812 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.757+0000 7fefc01f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-13T19:02:02.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-13T19:02:02.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:03.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:03.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:03.139 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:03.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:03.145 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:02.973+0000 7fefc01f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.145 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:03.091+0000 7fefc01f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.146 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:03.091+0000 7fefc01f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-13T19:02:03.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-13T19:02:03.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-13T19:02:03.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:03.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:03.436 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:03.437 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.437 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:03.437 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-13T19:02:03.437 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:03.475 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:03.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:03.481 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:03.307+0000 7fefc01f1640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.482 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:03.425+0000 7fefc01f1640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.482 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:03.425+0000 7fefc01f1640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:03.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:03.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:03.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:03.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:03.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:03.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:03.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:03.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:03.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:03.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:03.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:03.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:03.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:03.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:03.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:03.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:03.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:03.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:03.606 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:03.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:03.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:03.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:03.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:03.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:03.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:03.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:03.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:03.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:03.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:03.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:03.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:03.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:03.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:03.610 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:03.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:03.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:03.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:03.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:03.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:03.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:03.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:03.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:03.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:03.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.616 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:03.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:03.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:03.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:03.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:03.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:03.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:03.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-13T19:02:03.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:03.659 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:03.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:03.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:03.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:03.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:03.661 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:03.661 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:03.661 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:03.661 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:03.662 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.662 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.662 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:03.662 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:03.662 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.704 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.705 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:03.705 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:03.705 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:03.844 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.845 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:03.846 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:03.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:03.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-13T19:02:03.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:03.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:03.861 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:03.862 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.862 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:03.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:03.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:03.899 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:03.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-13T19:02:03.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:03.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:03.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eb035258-ee23-44c7-bfdf-7d2ecc96bc64 2026-02-13T19:02:03.907 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 eb035258-ee23-44c7-bfdf-7d2ecc96bc64 2026-02-13T19:02:03.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 eb035258-ee23-44c7-bfdf-7d2ecc96bc64' 2026-02-13T19:02:03.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:03.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArdY9pXj+HNhAAXvyWJwqE9wbgE9myZLPwgw== 2026-02-13T19:02:03.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArdY9pXj+HNhAAXvyWJwqE9wbgE9myZLPwgw=="}' 2026-02-13T19:02:03.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eb035258-ee23-44c7-bfdf-7d2ecc96bc64 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:04.017 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:04.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:04.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQArdY9pXj+HNhAAXvyWJwqE9wbgE9myZLPwgw== --osd-uuid eb035258-ee23-44c7-bfdf-7d2ecc96bc64 2026-02-13T19:02:04.040 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.039+0000 7f8c87e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.041 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.041+0000 7f8c87e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.042 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.042+0000 7f8c87e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.043 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.042+0000 7f8c87e29780 -1 bdev(0x344cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:04.043 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.042+0000 7f8c87e29780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:04.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:04.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:04.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:04.497 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:04.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:04.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:04.598 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:04.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-13T19:02:04.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:04.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:04.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:04.611 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.610+0000 7f88bee27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.612 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.611+0000 7f88bee27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.613 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.612+0000 7f88bee27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:04.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:04.699 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:04.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:04.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:04.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:04.839 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:04.837+0000 7f88bee27780 -1 Falling back to public interface 2026-02-13T19:02:05.302 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:05.301+0000 7f88bee27780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:05.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:05.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:05.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:05.795 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:05.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:05.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:05.894 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2550756999,v1:127.0.0.1:6801/2550756999] [v2:127.0.0.1:6802/2550756999,v1:127.0.0.1:6803/2550756999] exists,up eb035258-ee23-44c7-bfdf-7d2ecc96bc64 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:05.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:05.939 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:05.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:05.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:05.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:05.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:05.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:05.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:05.948 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:05.949 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:05.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:05.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:05.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:06.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:06.054 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:06.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:06.055 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:06.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:06.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:06.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:06.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:06.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:06.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:06.058 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:06.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:06.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:06.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:06.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:06.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:06.065 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:06.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:06.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:06.067 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:06.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:06.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:06.068 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:06.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:06.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:06.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:06.070 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:06.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:06.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:06.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:06.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:06.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.073 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:06.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:06.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:06.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:06.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:06.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:06.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-13T19:02:06.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:06.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:06.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:06.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:06.096 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.096 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:06.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-13T19:02:06.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:06.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:06.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:06.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:06.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:06.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:06.120 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:06.120 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:06.120 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:06.121 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:06.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:06.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:06.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:06.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:06.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:06.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:06.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:06.304 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:06.304 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:06.305 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:06.306 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:06.320 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-13T19:02:06.320 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:06.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:06.358 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:06.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:06.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:06.364 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:06.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=525a811f-c39a-4e1c-9449-7bd3596b3b36 2026-02-13T19:02:06.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 525a811f-c39a-4e1c-9449-7bd3596b3b36' 2026-02-13T19:02:06.365 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 525a811f-c39a-4e1c-9449-7bd3596b3b36 2026-02-13T19:02:06.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:06.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAudY9pL0RKFhAAhqX+5JSFSdhBsBfEz2KTQA== 2026-02-13T19:02:06.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAudY9pL0RKFhAAhqX+5JSFSdhBsBfEz2KTQA=="}' 2026-02-13T19:02:06.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 525a811f-c39a-4e1c-9449-7bd3596b3b36 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:06.477 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:06.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:06.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQAudY9pL0RKFhAAhqX+5JSFSdhBsBfEz2KTQA== --osd-uuid 525a811f-c39a-4e1c-9449-7bd3596b3b36 2026-02-13T19:02:06.500 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:06.498+0000 7fc337629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:06.501 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:06.500+0000 7fc337629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:06.501 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:06.501+0000 7fc337629780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:06.502 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:06.501+0000 7fc337629780 -1 bdev(0x3724c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:06.502 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:06.501+0000 7fc337629780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:06.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:06.947 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:06.947 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:06.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:06.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:07.050 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:07.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:07.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-13T19:02:07.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:07.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:07.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:07.065 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:07.064+0000 7f524d848780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:07.066 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:07.065+0000 7f524d848780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:07.067 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:07.066+0000 7f524d848780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:07.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:07.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:07.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:07.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:07.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:07.147 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:07.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:07.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:07.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:07.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:07.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:07.246 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:07.246+0000 7f524d848780 -1 Falling back to public interface 2026-02-13T19:02:07.637 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:07.636+0000 7f524d848780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:08.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:08.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:08.244 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:08.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:08.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:08.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:08.340 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2053001277,v1:127.0.0.1:6801/2053001277] [v2:127.0.0.1:6802/2053001277,v1:127.0.0.1:6803/2053001277] exists,up 525a811f-c39a-4e1c-9449-7bd3596b3b36 2026-02-13T19:02:08.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:08.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:08.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:08.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:08.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:08.386 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:08.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:08.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:08.394 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:08.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:08.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:08.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:08.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:08.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:08.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:08.503 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:08.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:08.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:08.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:08.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:08.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:08.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:08.506 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:08.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:08.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:08.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:08.511 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:08.511 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.511 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:08.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:08.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:08.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:08.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:08.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:08.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:08.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:08.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:08.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:08.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:08.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:08.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:08.518 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:08.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:08.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:08.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:08.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:08.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.521 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:08.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:08.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:08.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:08.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:08.523 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.523 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:08.524 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:08.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:08.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-13T19:02:08.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:08.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:08.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:08.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:08.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:08.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:08.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:08.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:08.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:08.567 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:08.568 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.568 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.568 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:08.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:08.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:08.611 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:08.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:08.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:08.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:08.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:08.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:08.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:08.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:08.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:08.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:08.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:08.748 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:08.748 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.749 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.749 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:08.749 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:08.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:08.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:08.765 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:08.803 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:08.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d' 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d 2026-02-13T19:02:08.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwdY9pJQ7LMBAA/xMXN0CM3ixWm6e98jlyyA== 2026-02-13T19:02:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwdY9pJQ7LMBAA/xMXN0CM3ixWm6e98jlyyA=="}' 2026-02-13T19:02:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:08.919 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:08.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:08.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQAwdY9pJQ7LMBAA/xMXN0CM3ixWm6e98jlyyA== --osd-uuid 8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d 2026-02-13T19:02:08.941 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:08.940+0000 7f09ba1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:08.942 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:08.941+0000 7f09ba1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:08.943 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:08.942+0000 7f09ba1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:08.943 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:08.942+0000 7f09ba1d8780 -1 bdev(0x2712c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:08.944 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:08.942+0000 7f09ba1d8780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:09.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:09.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:09.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:09.335 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:09.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:09.437 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:09.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:09.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-13T19:02:09.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:09.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:09.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:09.452 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:09.451+0000 7f6a61238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:09.453 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:09.452+0000 7f6a61238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:09.454 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:09.453+0000 7f6a61238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:09.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:09.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:09.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:09.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:09.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:09.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:09.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:09.537 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:09.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:09.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:09.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:09.720 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:09.720+0000 7f6a61238780 -1 Falling back to public interface 2026-02-13T19:02:10.055 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:10.054+0000 7f6a61238780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:10.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:10.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:10.633 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:10.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:10.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:10.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:10.728 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3737233841,v1:127.0.0.1:6801/3737233841] [v2:127.0.0.1:6802/3737233841,v1:127.0.0.1:6803/3737233841] exists,up 8ac992a1-e5b7-4c3d-a7b1-6ad6f094b65d 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:10.730 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.731 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:10.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:10.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:10.776 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:10.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:10.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:10.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:10.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:10.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:10.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:10.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:10.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:10.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:10.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:10.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:10.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:10.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:10.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:10.895 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:10.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:10.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:10.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:10.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:10.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:10.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:10.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:10.903 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:10.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:10.904 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:10.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:10.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:10.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:10.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:10.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:10.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:10.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:10.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:10.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:10.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:10.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:10.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:10.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:10.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:10.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:10.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:10.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:10.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:10.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:10.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:10.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:10.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-13T19:02:10.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:10.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:10.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:10.932 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:10.932 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.932 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.932 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:10.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-13T19:02:10.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:10.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:10.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:10.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:10.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:10.954 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:10.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:11.000 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:11.000 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:11.000 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:11.001 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:11.002 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:11.002 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:11.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:11.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:11.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:11.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:11.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:11.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:11.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:11.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:11.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:11.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:11.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:11.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-13T19:02:11.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:11.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:11.196 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:11.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:11.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b916c51a-ed74-4623-a5b5-6cdf4961b917 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b916c51a-ed74-4623-a5b5-6cdf4961b917' 2026-02-13T19:02:11.207 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 b916c51a-ed74-4623-a5b5-6cdf4961b917 2026-02-13T19:02:11.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:11.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAzdY9pfhGtDBAA4VV/Y2O+kSPxPGbG79fc4g== 2026-02-13T19:02:11.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAzdY9pfhGtDBAA4VV/Y2O+kSPxPGbG79fc4g=="}' 2026-02-13T19:02:11.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b916c51a-ed74-4623-a5b5-6cdf4961b917 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:11.313 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:11.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:11.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQAzdY9pfhGtDBAA4VV/Y2O+kSPxPGbG79fc4g== --osd-uuid b916c51a-ed74-4623-a5b5-6cdf4961b917 2026-02-13T19:02:11.337 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.336+0000 7f566fa57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.338 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.337+0000 7f566fa57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.339 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.338+0000 7f566fa57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.339 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.338+0000 7f566fa57780 -1 bdev(0x2a90c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:11.340 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.338+0000 7f566fa57780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:11.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:11.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:11.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:11.733 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:11.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:11.833 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:11.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:11.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-13T19:02:11.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:11.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:11.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:11.848 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.847+0000 7f354b05d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.850 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.849+0000 7f354b05d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.850 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:11.850+0000 7f354b05d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:11.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:12.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:12.057 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:12.057+0000 7f354b05d780 -1 Falling back to public interface 2026-02-13T19:02:12.423 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:12.421+0000 7f354b05d780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:13.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:13.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:13.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:13.029 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:13.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:13.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:13.126 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2266608344,v1:127.0.0.1:6801/2266608344] [v2:127.0.0.1:6802/2266608344,v1:127.0.0.1:6803/2266608344] exists,up b916c51a-ed74-4623-a5b5-6cdf4961b917 2026-02-13T19:02:13.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:13.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-13T19:02:13.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:13.173 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:13.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:13.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:13.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:13.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:13.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:13.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:13.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:13.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:13.289 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:13.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:13.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:13.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:13.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:13.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:13.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:13.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:13.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:13.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:13.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:13.296 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:13.296 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.296 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:13.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:13.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:13.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:13.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:13.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:13.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:13.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:13.302 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:13.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:13.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:13.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:13.303 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:13.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:13.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:13.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:13.306 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:13.306 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.306 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.306 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:13.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:13.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:13.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:13.308 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:13.308 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.308 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:13.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:13.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:13.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:13.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:13.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-13T19:02:13.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:13.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:13.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:13.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:13.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:13.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:02:13.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:13.353 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:13.354 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:13.354 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:13.354 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:13.355 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.355 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.355 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:13.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:13.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:13.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:13.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:13.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:13.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:13.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:13.401 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:13.401 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:13.401 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:13.401 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:13.401 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.402 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.402 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:13.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:13.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.444 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:13.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-13T19:02:13.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:13.483 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:13.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.588 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:13.588 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:13.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:13.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b0b07ae7-8653-4567-a337-72abb358c223 2026-02-13T19:02:13.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b0b07ae7-8653-4567-a337-72abb358c223' 2026-02-13T19:02:13.610 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 b0b07ae7-8653-4567-a337-72abb358c223 2026-02-13T19:02:13.610 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:13.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA1dY9p69GdJBAAIIbP/XeaPYT86oHF3v2x5w== 2026-02-13T19:02:13.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA1dY9p69GdJBAAIIbP/XeaPYT86oHF3v2x5w=="}' 2026-02-13T19:02:13.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b0b07ae7-8653-4567-a337-72abb358c223 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:13.716 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:13.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:13.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA1dY9p69GdJBAAIIbP/XeaPYT86oHF3v2x5w== --osd-uuid b0b07ae7-8653-4567-a337-72abb358c223 2026-02-13T19:02:13.738 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:13.737+0000 7f3d5e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:13.739 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:13.738+0000 7f3d5e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:13.740 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:13.739+0000 7f3d5e262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:13.741 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:13.740+0000 7f3d5e262780 -1 bdev(0x2c8ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:13.741 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:13.740+0000 7f3d5e262780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:14.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:14.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:14.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:14.155 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:14.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:14.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:14.259 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:14.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:14.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:14.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:14.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:14.274 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:14.273+0000 7f9c15c15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:14.275 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:14.275+0000 7f9c15c15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:14.276 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:14.275+0000 7f9c15c15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:14.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:14.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:14.358 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:14.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:14.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:14.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:14.494 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:14.493+0000 7f9c15c15780 -1 Falling back to public interface 2026-02-13T19:02:14.797 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:14.796+0000 7f9c15c15780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:15.455 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:15.455 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:15.456 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:15.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:15.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:15.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:15.551 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3282816618,v1:127.0.0.1:6801/3282816618] [v2:127.0.0.1:6802/3282816618,v1:127.0.0.1:6803/3282816618] exists,up b0b07ae7-8653-4567-a337-72abb358c223 2026-02-13T19:02:15.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:15.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:15.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-13T19:02:15.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:15.600 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:15.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:15.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:15.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-13T19:02:15.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:15.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:15.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:15.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:15.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:15.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:15.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:15.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:15.718 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:15.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:15.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:15.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:15.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:15.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:15.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:15.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:15.725 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:15.725 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.725 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:15.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:15.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:15.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:15.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:15.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:15.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:15.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:15.757 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:15.757 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:15.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:15.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:15.784 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.785 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.785 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:15.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:15.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:15.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:15.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:15.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:15.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:15.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:15.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:15.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:15.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:15.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:15.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:15.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:15.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:15.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:15.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:15.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:15.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:16.024 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:16.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:16.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:16.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=66226ef3-a074-4e85-9304-baaa529792b0 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 66226ef3-a074-4e85-9304-baaa529792b0' 2026-02-13T19:02:16.036 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 66226ef3-a074-4e85-9304-baaa529792b0 2026-02-13T19:02:16.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:16.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA4dY9pNj9bAhAAqEGn3fiacEcATed/32BgNA== 2026-02-13T19:02:16.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA4dY9pNj9bAhAAqEGn3fiacEcATed/32BgNA=="}' 2026-02-13T19:02:16.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 66226ef3-a074-4e85-9304-baaa529792b0 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:16.141 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:16.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:16.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQA4dY9pNj9bAhAAqEGn3fiacEcATed/32BgNA== --osd-uuid 66226ef3-a074-4e85-9304-baaa529792b0 2026-02-13T19:02:16.164 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.162+0000 7f399ce3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.165 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.164+0000 7f399ce3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.166 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.165+0000 7f399ce3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.167 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.165+0000 7f399ce3f780 -1 bdev(0x293cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:16.167 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.165+0000 7f399ce3f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:16.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:16.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:16.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:16.594 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:16.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:16.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:16.710 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.710+0000 7f1465a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.712 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.711+0000 7f1465a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.713 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.712+0000 7f1465a18780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:16.792 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:16.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:16.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:16.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:16.913 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:16.911+0000 7f1465a18780 -1 Falling back to public interface 2026-02-13T19:02:17.207 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:17.205+0000 7f1465a18780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:17.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:17.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:17.892 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:17.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:17.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:17.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:17.987 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2607543124,v1:127.0.0.1:6801/2607543124] [v2:127.0.0.1:6802/2607543124,v1:127.0.0.1:6803/2607543124] exists,up 66226ef3-a074-4e85-9304-baaa529792b0 2026-02-13T19:02:17.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:17.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:17.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:17.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:17.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:17.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:18.031 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:18.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:18.036 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:15.773+0000 7f8199050c00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:18.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:18.037 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:16.912+0000 7f1465a18780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:18.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:18.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:18.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:18.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:18.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:18.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:18.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:18.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:18.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:18.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:18.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:18.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:18.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:18.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:18.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:18.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:18.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:18.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:18.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:18.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:18.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:18.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:18.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:18.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:18.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:18.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:18.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:18.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:18.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:18.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:18.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:18.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:18.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:18.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:18.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:18.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:18.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:18.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:18.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:18.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:18.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:18.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:18.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:18.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:18.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:18.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:18.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:18.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:18.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:18.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:18.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:18.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:18.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:18.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:18.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:18.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:18.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:18.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:18.211 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:18.212 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:18.212 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.212 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.212 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:18.212 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:18.213 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:18.254 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:18.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:18.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:18.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:18.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:18.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:18.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:18.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:18.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:18.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:18.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:18.398 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:18.398 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.399 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:18.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:18.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:18.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:18.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:18.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:18.414 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:18.414 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.415 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:18.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:18.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:18.451 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:18.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:18.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:18.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:18.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:18.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8ccce7e3-f891-4d70-aaa6-54890df3e4aa 2026-02-13T19:02:18.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8ccce7e3-f891-4d70-aaa6-54890df3e4aa' 2026-02-13T19:02:18.462 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 8ccce7e3-f891-4d70-aaa6-54890df3e4aa 2026-02-13T19:02:18.463 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:18.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA6dY9pXaXQGxAAGfLZZe+4qL/3guEquP3ymw== 2026-02-13T19:02:18.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA6dY9pXaXQGxAAGfLZZe+4qL/3guEquP3ymw=="}' 2026-02-13T19:02:18.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8ccce7e3-f891-4d70-aaa6-54890df3e4aa -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:18.570 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:18.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:18.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQA6dY9pXaXQGxAAGfLZZe+4qL/3guEquP3ymw== --osd-uuid 8ccce7e3-f891-4d70-aaa6-54890df3e4aa 2026-02-13T19:02:18.592 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:18.591+0000 7f70ddc15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:18.594 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:18.593+0000 7f70ddc15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:18.595 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:18.594+0000 7f70ddc15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:18.595 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:18.594+0000 7f70ddc15780 -1 bdev(0x2702c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:18.595 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:18.594+0000 7f70ddc15780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:19.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:19.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:19.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:19.024 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:19.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:19.130 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:19.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-13T19:02:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:19.144 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:19.142+0000 7fbb8fc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:19.145 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:19.143+0000 7fbb8fc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:19.146 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:19.144+0000 7fbb8fc3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:19.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:19.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:19.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:19.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:19.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:19.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:19.229 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:19.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:19.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:19.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:19.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:19.356 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:19.354+0000 7fbb8fc3d780 -1 Falling back to public interface 2026-02-13T19:02:19.752 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:19.749+0000 7fbb8fc3d780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:20.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:20.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:20.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:20.326 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:20.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:20.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:20.422 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1391542693,v1:127.0.0.1:6801/1391542693] [v2:127.0.0.1:6802/1391542693,v1:127.0.0.1:6803/1391542693] exists,up 8ccce7e3-f891-4d70-aaa6-54890df3e4aa 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:20.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:20.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:20.467 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:20.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:20.472 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:18.200+0000 7fc72b93dc00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:20.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:20.473 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:19.354+0000 7fbb8fc3d780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:20.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:20.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:20.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:20.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:20.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:20.582 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:20.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:20.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:20.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:20.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:20.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:20.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:20.585 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:20.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:20.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:20.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:20.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:20.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:20.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:20.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:20.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:20.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:20.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:20.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:20.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:20.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:20.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:20.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:20.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:20.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:20.597 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:20.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:20.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:20.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:20.599 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:20.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:20.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:20.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:20.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:20.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:20.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:20.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:20.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:20.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:20.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:20.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:20.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:20.622 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:20.622 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.623 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.623 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:20.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:20.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:20.647 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:20.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:20.693 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:20.694 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.694 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.694 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:20.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:20.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:20.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:20.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:20.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:20.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:20.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:20.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:20.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:20.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:20.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:20.836 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:20.836 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.836 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:20.837 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:20.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:20.851 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:20.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:20.891 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:20.900 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:20.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=89ee316a-7097-49d0-8197-814cd3b22534 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 89ee316a-7097-49d0-8197-814cd3b22534' 2026-02-13T19:02:20.902 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 89ee316a-7097-49d0-8197-814cd3b22534 2026-02-13T19:02:20.903 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:20.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA8dY9pyO/4NRAAknEWzjII2sLxSvcsCYPswQ== 2026-02-13T19:02:20.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA8dY9pyO/4NRAAknEWzjII2sLxSvcsCYPswQ=="}' 2026-02-13T19:02:20.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 89ee316a-7097-49d0-8197-814cd3b22534 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:21.009 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:21.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:21.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQA8dY9pyO/4NRAAknEWzjII2sLxSvcsCYPswQ== --osd-uuid 89ee316a-7097-49d0-8197-814cd3b22534 2026-02-13T19:02:21.032 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.030+0000 7f9c47c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.033 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.031+0000 7f9c47c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.034 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.032+0000 7f9c47c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.033+0000 7f9c47c2d780 -1 bdev(0x342ec00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:21.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.033+0000 7f9c47c2d780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:21.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:21.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:21.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:21.429 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:21.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:21.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:21.532 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-13T19:02:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:21.546 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.544+0000 7f43737c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.547 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.546+0000 7f43737c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.548 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.547+0000 7f43737c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:21.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:21.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:21.793 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:21.791+0000 7f43737c6780 -1 Falling back to public interface 2026-02-13T19:02:22.133 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:22.131+0000 7f43737c6780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:22.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:22.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:22.726 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:22.825 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2466940774,v1:127.0.0.1:6801/2466940774] [v2:127.0.0.1:6802/2466940774,v1:127.0.0.1:6803/2466940774] exists,up 89ee316a-7097-49d0-8197-814cd3b22534 2026-02-13T19:02:22.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:22.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:22.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:22.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:22.872 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:22.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:22.877 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:20.635+0000 7f0d706fdc00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:22.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:22.878 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:21.791+0000 7f43737c6780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:22.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:22.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:22.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:22.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:22.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:22.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:22.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:22.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:22.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:22.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:22.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:22.991 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:22.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:22.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:22.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:22.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:22.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:22.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:22.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:22.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:22.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:23.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:23.000 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:23.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:23.001 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:23.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:23.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:23.002 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:23.002 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:23.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:23.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:23.003 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:23.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:23.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:23.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:23.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:23.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:23.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:23.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:23.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:23.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:23.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:23.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:23.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:23.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:23.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:23.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:23.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:23.051 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:23.052 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:23.052 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.053 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.053 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:23.053 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:23.053 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:23.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:23.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:23.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:23.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:23.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:23.233 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:23.233 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:23.233 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:23.233 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:23.233 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.234 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.234 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:23.234 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:23.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.249 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.250 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:23.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:23.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:23.287 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:23.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:23.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:23.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:23.296 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:23.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9d8e5e44-81b9-4d61-94c0-7eaf07f52379 2026-02-13T19:02:23.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9d8e5e44-81b9-4d61-94c0-7eaf07f52379' 2026-02-13T19:02:23.297 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 9d8e5e44-81b9-4d61-94c0-7eaf07f52379 2026-02-13T19:02:23.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:23.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/dY9pP2UVEhAAwOITpHKWXDuuAwnNFWvprg== 2026-02-13T19:02:23.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/dY9pP2UVEhAAwOITpHKWXDuuAwnNFWvprg=="}' 2026-02-13T19:02:23.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9d8e5e44-81b9-4d61-94c0-7eaf07f52379 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:23.405 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:23.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:23.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQA/dY9pP2UVEhAAwOITpHKWXDuuAwnNFWvprg== --osd-uuid 9d8e5e44-81b9-4d61-94c0-7eaf07f52379 2026-02-13T19:02:23.427 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.425+0000 7f7864a6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:23.428 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.427+0000 7f7864a6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:23.429 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.427+0000 7f7864a6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:23.430 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.428+0000 7f7864a6f780 -1 bdev(0x28eec00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:23.430 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.428+0000 7f7864a6f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:23.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:23.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:23.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:23.809 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:23.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:23.911 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:23.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:23.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-13T19:02:23.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:23.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:23.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:23.925 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.923+0000 7f6092a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:23.927 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.925+0000 7f6092a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:23.927 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:23.926+0000 7f6092a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:24.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:24.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:24.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:24.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:24.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:24.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:24.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:24.011 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:24.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:24.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:24.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:24.134 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:24.132+0000 7f6092a29780 -1 Falling back to public interface 2026-02-13T19:02:24.442 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:24.440+0000 7f6092a29780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:25.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:25.203 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/249892050,v1:127.0.0.1:6801/249892050] [v2:127.0.0.1:6802/249892050,v1:127.0.0.1:6803/249892050] exists,up 9d8e5e44-81b9-4d61-94c0-7eaf07f52379 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:25.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:25.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:25.250 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:25.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:25.255 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:23.040+0000 7f78a8c0fc00 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:25.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:25.257 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:24.133+0000 7f6092a29780 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:25.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:25.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:25.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:25.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:25.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:25.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:25.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:25.368 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:25.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:25.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:25.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:25.370 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:25.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:25.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:25.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:25.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:25.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:25.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:25.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:25.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:25.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:25.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:25.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:25.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:25.378 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:25.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:25.379 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:25.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:25.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:25.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:25.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:25.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:25.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:25.381 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:25.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:25.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:25.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:25.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:25.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:25.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:25.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:25.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:25.386 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:25.386 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.386 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:25.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:25.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:25.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:25.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:25.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:25.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:25.405 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:25.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:25.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:25.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:25.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:25.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:25.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:25.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:25.430 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:25.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:25.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.473 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.474 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:25.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:25.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:25.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:25.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:25.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:25.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:25.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:25.669 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:25.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:25.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:25.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:25.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:25.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:25.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5ac779aa-e526-4ca5-b9fd-ae5779434672 2026-02-13T19:02:25.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5ac779aa-e526-4ca5-b9fd-ae5779434672' 2026-02-13T19:02:25.680 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 5ac779aa-e526-4ca5-b9fd-ae5779434672 2026-02-13T19:02:25.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:25.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBBdY9pt9TIKBAAADXKvhNF6+cXNjf+9/ZP/g== 2026-02-13T19:02:25.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBBdY9pt9TIKBAAADXKvhNF6+cXNjf+9/ZP/g=="}' 2026-02-13T19:02:25.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5ac779aa-e526-4ca5-b9fd-ae5779434672 -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:25.784 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:25.790 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:25.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQBBdY9pt9TIKBAAADXKvhNF6+cXNjf+9/ZP/g== --osd-uuid 5ac779aa-e526-4ca5-b9fd-ae5779434672 2026-02-13T19:02:25.806 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:25.805+0000 7f7a131c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:25.808 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:25.806+0000 7f7a131c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:25.809 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:25.807+0000 7f7a131c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:25.809 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:25.807+0000 7f7a131c6780 -1 bdev(0x35f4c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:25.809 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:25.807+0000 7f7a131c6780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:26.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:26.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:26.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:26.216 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:26.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:26.317 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-13T19:02:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:26.332 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:26.331+0000 7ffb18bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:26.334 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:26.332+0000 7ffb18bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:26.334 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:26.333+0000 7ffb18bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:26.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:26.417 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:26.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:26.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:26.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:26.521 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:26.519+0000 7ffb18bd6780 -1 Falling back to public interface 2026-02-13T19:02:26.838 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:26.836+0000 7ffb18bd6780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:27.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:27.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:27.518 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:27.613 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3310412742,v1:127.0.0.1:6801/3310412742] [v2:127.0.0.1:6802/3310412742,v1:127.0.0.1:6803/3310412742] exists,up 5ac779aa-e526-4ca5-b9fd-ae5779434672 2026-02-13T19:02:27.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:27.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:27.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:27.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:27.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:27.661 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:27.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:27.667 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:25.418+0000 7f0488214c00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:27.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:27.668 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:26.520+0000 7ffb18bd6780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:27.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:27.775 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:27.775 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:27.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:27.777 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:27.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:27.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:27.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:27.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:27.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:27.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:27.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:27.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:27.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:27.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:27.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:27.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:27.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:27.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:27.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:27.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:27.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:27.790 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:27.790 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:27.791 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:27.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:27.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:27.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:27.792 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:27.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:27.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:27.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:27.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:27.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:27.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:27.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:27.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-13T19:02:27.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:27.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.35854 2026-02-13T19:02:27.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:27.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:27.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:27.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:27.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-13T19:02:27.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:27.817 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:27.817 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:27.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:27.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:27.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:27.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:27.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:27.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:27.843 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:27.843 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:27.843 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:27.843 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:27.844 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get fsid 2026-02-13T19:02:27.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:27.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:27.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:27.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:27.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:27.889 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:27.889 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:27.889 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:27.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.35854/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:27.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-13T19:02:27.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:27.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:27.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:27.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:27.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-13T19:02:27.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:28.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:28.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:28.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:28.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:28.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:28.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:28.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:28.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:28.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:28.048 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-mon.a.asok 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:28.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-mon.a.asok log flush 2026-02-13T19:02:28.088 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:28.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' 2026-02-13T19:02:28.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-13T19:02:28.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:28.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa 2026-02-13T19:02:28.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa' 2026-02-13T19:02:28.095 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa 2026-02-13T19:02:28.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:28.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEdY9p+k4rBhAAv27Q904c6r9CjbhxXWQPkw== 2026-02-13T19:02:28.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEdY9p+k4rBhAAv27Q904c6r9CjbhxXWQPkw=="}' 2026-02-13T19:02:28.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa -i td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:28.208 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:28.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-13T19:02:28.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQBEdY9p+k4rBhAAv27Q904c6r9CjbhxXWQPkw== --osd-uuid 25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa 2026-02-13T19:02:28.232 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.230+0000 7f6050271780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.233 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.231+0000 7f6050271780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.234 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.232+0000 7f6050271780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.234 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.232+0000 7f6050271780 -1 bdev(0x3c6cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:28.235 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.233+0000 7f6050271780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-13T19:02:28.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-13T19:02:28.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:28.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:28.641 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:28.641 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:28.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:28.741 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:28.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4c76c020-0c54-49e2-80e3-9737cc9e63f6 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.35854/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-13T19:02:28.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:28.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:28.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:28.755 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.753+0000 7f7778234780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.756 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.755+0000 7f7778234780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.757 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.755+0000 7f7778234780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:28.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:28.837 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:28.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:28.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:28.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:28.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:28.972 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:28.970+0000 7f7778234780 -1 Falling back to public interface 2026-02-13T19:02:29.376 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:29.374+0000 7f7778234780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:29.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:29.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:29.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:29.934 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:29.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:29.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:30.027 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2167186686,v1:127.0.0.1:6801/2167186686] [v2:127.0.0.1:6802/2167186686,v1:127.0.0.1:6803/2167186686] exists,up 25be5d85-0cc7-4f9f-be3f-dc16f0f39eaa 2026-02-13T19:02:30.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:30.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:30.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.35854/ceph-osd.0.asok 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-13T19:02:30.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.35854/ceph-osd.0.asok log flush 2026-02-13T19:02:30.075 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:30.080 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-13T19:02:30.081 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:27.831+0000 7f91ff85dc00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:30.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-13T19:02:30.082 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:28.970+0000 7f7778234780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:30.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:30.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:30.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:30.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:30.190 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:30.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:30.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:30.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:30.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:30.193 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:30.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:30.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:30.195 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:30.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:30.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:30.199 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:30.199 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.199 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:30.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:30.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:30.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code-plugins 0 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:30.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:30.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:30.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:30.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:30.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:30.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:30.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:30.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:30.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:30.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:30.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:30.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:02:30.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-13T19:02:30.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:30.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.35854 2026-02-13T19:02:30.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.35854 2026-02-13T19:02:30.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:30.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:30.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:02:30.211 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:02:30.211 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:02:30.276 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-13T19:02:30.276 DEBUG:teuthology.orchestra.run.trial034:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-13T19:02:30.339 INFO:tasks.workunit.client.0.trial034.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:02:30.345 INFO:tasks.workunit.client.0.trial034.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:02:30.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-13T19:02:30.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none ' 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-13T19:02:30.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:02:30.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:02:30.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:02:30.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:02:30.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:02:30.349 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:02:30.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:02:30.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:02:30.350 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:02:30.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:02:30.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:02:30.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-13T19:02:30.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:02:30.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.66758 2026-02-13T19:02:30.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:02:30.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:02:30.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-13T19:02:30.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:02:30.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.66758 2026-02-13T19:02:30.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-13T19:02:30.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-13T19:02:30.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:02:30.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:02:30.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:02:30.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-13T19:02:30.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-13T19:02:30.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:02:30.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:30.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:30.375 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:30.375 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.375 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:30.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:02:30.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:02:30.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.399 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.66758/ceph-mon.a.asok 2026-02-13T19:02:30.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:30.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.66758/ceph-mon.a.asok config get fsid 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:30.442 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:30.443 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.443 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.443 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.66758/ceph-mon.a.asok 2026-02-13T19:02:30.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:02:30.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.66758/ceph-mon.a.asok config get mon_host 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-13T19:02:30.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.587 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.588 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:30.588 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:30.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:02:30.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.66758/ceph-mon.a.asok 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-13T19:02:30.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.66758/ceph-mon.a.asok log flush 2026-02-13T19:02:30.643 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:02:30.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-13T19:02:30.647 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:30.388+0000 7f7c93838c00 0 load: jerasure load: lrc 2026-02-13T19:02:30.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-13T19:02:30.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:30.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-13T19:02:30.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:30.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:30.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-13T19:02:30.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:30.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9197dc72-68da-4532-90fe-9a8d4da6f7c6 2026-02-13T19:02:30.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9197dc72-68da-4532-90fe-9a8d4da6f7c6' 2026-02-13T19:02:30.655 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 9197dc72-68da-4532-90fe-9a8d4da6f7c6 2026-02-13T19:02:30.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:30.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGdY9pbsRgJxAAf0pLIVFKISA9tgxC27X+LQ== 2026-02-13T19:02:30.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGdY9pbsRgJxAAf0pLIVFKISA9tgxC27X+LQ=="}' 2026-02-13T19:02:30.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9197dc72-68da-4532-90fe-9a8d4da6f7c6 -i td/test-erasure-code/0/new.json 2026-02-13T19:02:30.763 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:30.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-13T19:02:30.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBGdY9pbsRgJxAAf0pLIVFKISA9tgxC27X+LQ== --osd-uuid 9197dc72-68da-4532-90fe-9a8d4da6f7c6 2026-02-13T19:02:30.787 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:30.785+0000 7fd2e1c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:30.789 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:30.787+0000 7fd2e1c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:30.789 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:30.787+0000 7fd2e1c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:30.790 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:30.788+0000 7fd2e1c2c780 -1 bdev(0x3a82c00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:30.790 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:30.788+0000 7fd2e1c2c780 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-13T19:02:31.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-13T19:02:31.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:31.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:02:31.216 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:02:31.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:31.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:02:31.320 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:02:31.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:31.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:31.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:31.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:31.335 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:31.333+0000 7f1ddf03d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:31.336 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:31.334+0000 7f1ddf03d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:31.337 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:31.335+0000 7f1ddf03d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:31.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:02:31.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:31.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:02:31.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:31.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:31.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:31.416 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:31.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:31.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:31.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:31.504 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:31.502+0000 7f1ddf03d780 -1 Falling back to public interface 2026-02-13T19:02:31.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:31.808 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:31.806+0000 7f1ddf03d780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:02:32.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:32.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:32.517 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:32.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:32.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:32.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:02:32.613 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3327720883,v1:127.0.0.1:6801/3327720883] [v2:127.0.0.1:6802/3327720883,v1:127.0.0.1:6803/3327720883] exists,up 9197dc72-68da-4532-90fe-9a8d4da6f7c6 2026-02-13T19:02:32.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:32.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:32.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:32.621 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:32.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:32.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-13T19:02:32.624 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:32.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b42f6935-8b0d-4c39-b2b6-6f0597867319 2026-02-13T19:02:32.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b42f6935-8b0d-4c39-b2b6-6f0597867319' 2026-02-13T19:02:32.624 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 b42f6935-8b0d-4c39-b2b6-6f0597867319 2026-02-13T19:02:32.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:32.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIdY9pwvFQJRAAynYm34NOrWDIegYKf5Qezg== 2026-02-13T19:02:32.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIdY9pwvFQJRAAynYm34NOrWDIegYKf5Qezg=="}' 2026-02-13T19:02:32.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b42f6935-8b0d-4c39-b2b6-6f0597867319 -i td/test-erasure-code/1/new.json 2026-02-13T19:02:32.731 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:32.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-13T19:02:32.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIdY9pwvFQJRAAynYm34NOrWDIegYKf5Qezg== --osd-uuid b42f6935-8b0d-4c39-b2b6-6f0597867319 2026-02-13T19:02:32.753 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:32.751+0000 7f6021a16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:32.754 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:32.752+0000 7f6021a16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:32.755 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:32.753+0000 7f6021a16780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:32.755 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:32.754+0000 7f6021a16780 -1 bdev(0x2c72c00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:32.756 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:32.754+0000 7f6021a16780 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-13T19:02:33.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-13T19:02:33.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:33.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:02:33.085 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:02:33.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:33.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:02:33.404 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:02:33.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:33.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:33.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:33.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:33.419 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:33.417+0000 7efd7ae44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:33.420 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:33.418+0000 7efd7ae44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:33.421 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:33.419+0000 7efd7ae44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:33.551 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:33.549+0000 7efd7ae44780 -1 Falling back to public interface 2026-02-13T19:02:33.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:02:33.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:33.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:02:33.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:33.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:33.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:33.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:33.637 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:33.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:33.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:02:33.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:33.878 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:33.876+0000 7efd7ae44780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:02:34.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:34.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:34.862 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:34.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:34.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:34.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:02:35.092 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3718051237,v1:127.0.0.1:6811/3718051237] [v2:127.0.0.1:6812/3718051237,v1:127.0.0.1:6813/3718051237] exists,up b42f6935-8b0d-4c39-b2b6-6f0597867319 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:02:35.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:35.097 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:35.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:35.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=85201380-e720-408b-8cb2-ec0197ec0169 2026-02-13T19:02:35.100 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 85201380-e720-408b-8cb2-ec0197ec0169 2026-02-13T19:02:35.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 85201380-e720-408b-8cb2-ec0197ec0169' 2026-02-13T19:02:35.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:35.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLdY9p3cNIBhAA2AhznRsRj5WYmd1FBzYSHg== 2026-02-13T19:02:35.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLdY9p3cNIBhAA2AhznRsRj5WYmd1FBzYSHg=="}' 2026-02-13T19:02:35.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 85201380-e720-408b-8cb2-ec0197ec0169 -i td/test-erasure-code/2/new.json 2026-02-13T19:02:35.335 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:02:35.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-13T19:02:35.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLdY9p3cNIBhAA2AhznRsRj5WYmd1FBzYSHg== --osd-uuid 85201380-e720-408b-8cb2-ec0197ec0169 2026-02-13T19:02:35.359 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:35.357+0000 7fe51b22e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:35.361 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:35.359+0000 7fe51b22e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:35.362 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:35.360+0000 7fe51b22e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:35.362 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:35.360+0000 7fe51b22e780 -1 bdev(0x2e24c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:35.362 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:35.360+0000 7fe51b22e780 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-13T19:02:35.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-13T19:02:35.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:35.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:02:35.695 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:02:35.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:36.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:02:36.010 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:02:36.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:36.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:36.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:36.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:36.025 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:36.023+0000 7f7aa6e4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:36.026 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:36.024+0000 7f7aa6e4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:36.027 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:36.025+0000 7f7aa6e4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:36.156 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:36.154+0000 7f7aa6e4a780 -1 Falling back to public interface 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:02:36.463 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:36.462+0000 7f7aa6e4a780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:02:36.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:37.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:37.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:37.479 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:37.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:37.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:37.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:02:37.709 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1195282145,v1:127.0.0.1:6819/1195282145] [v2:127.0.0.1:6820/1195282145,v1:127.0.0.1:6821/1195282145] exists,up 85201380-e720-408b-8cb2-ec0197ec0169 2026-02-13T19:02:37.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:37.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:37.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:37.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:37.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:37.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-13T19:02:37.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:37.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=364600fc-dc7e-4e02-9179-dbb88e787609 2026-02-13T19:02:37.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 364600fc-dc7e-4e02-9179-dbb88e787609' 2026-02-13T19:02:37.717 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 364600fc-dc7e-4e02-9179-dbb88e787609 2026-02-13T19:02:37.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:37.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBNdY9pYLsRKxAAWRKqepzEbwM5uxK8K93fEw== 2026-02-13T19:02:37.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBNdY9pYLsRKxAAWRKqepzEbwM5uxK8K93fEw=="}' 2026-02-13T19:02:37.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 364600fc-dc7e-4e02-9179-dbb88e787609 -i td/test-erasure-code/3/new.json 2026-02-13T19:02:37.953 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:02:37.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-13T19:02:37.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBNdY9pYLsRKxAAWRKqepzEbwM5uxK8K93fEw== --osd-uuid 364600fc-dc7e-4e02-9179-dbb88e787609 2026-02-13T19:02:37.977 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:37.977+0000 7fc08ea32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:37.979 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:37.978+0000 7fc08ea32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:37.980 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:37.979+0000 7fc08ea32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:37.980 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:37.979+0000 7fc08ea32780 -1 bdev(0x3040c00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:37.980 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:37.979+0000 7fc08ea32780 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-13T19:02:38.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-13T19:02:38.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:38.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:02:38.315 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:02:38.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:38.638 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:02:38.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:02:38.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:38.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:38.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:38.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:38.657 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:38.656+0000 7f983423f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:38.659 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:38.658+0000 7f983423f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:38.659 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:38.659+0000 7f983423f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:38.793 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:38.791+0000 7f983423f780 -1 Falling back to public interface 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:38.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:38.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:38.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:02:39.085 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:39.084+0000 7f983423f780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:02:39.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:40.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:40.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:40.102 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:40.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:40.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:40.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:02:40.329 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1366531641,v1:127.0.0.1:6827/1366531641] [v2:127.0.0.1:6828/1366531641,v1:127.0.0.1:6829/1366531641] exists,up 364600fc-dc7e-4e02-9179-dbb88e787609 2026-02-13T19:02:40.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:40.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:40.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:40.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:40.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-13T19:02:40.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:40.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-13T19:02:40.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:40.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1 2026-02-13T19:02:40.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1' 2026-02-13T19:02:40.336 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1 2026-02-13T19:02:40.337 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:40.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBQdY9pab1bFBAAz3qLjSrqfHEPhkfvO3mq9A== 2026-02-13T19:02:40.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBQdY9pab1bFBAAz3qLjSrqfHEPhkfvO3mq9A=="}' 2026-02-13T19:02:40.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1 -i td/test-erasure-code/4/new.json 2026-02-13T19:02:40.571 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:02:40.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-13T19:02:40.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBQdY9pab1bFBAAz3qLjSrqfHEPhkfvO3mq9A== --osd-uuid 9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1 2026-02-13T19:02:40.595 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:40.594+0000 7f96e7423780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:40.597 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:40.596+0000 7f96e7423780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:40.598 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:40.597+0000 7f96e7423780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:40.598 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:40.597+0000 7f96e7423780 -1 bdev(0x2e7ec00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:40.598 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:40.597+0000 7f96e7423780 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-13T19:02:40.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-13T19:02:40.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:40.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:02:40.938 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:02:40.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:41.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:02:41.256 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:02:41.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:41.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:41.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:41.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:41.271 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:41.270+0000 7f4fae412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:41.272 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:41.271+0000 7f4fae412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:41.273 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:41.272+0000 7f4fae412780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:41.418 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:41.417+0000 7f4fae412780 -1 Falling back to public interface 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:41.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:41.487 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:41.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:41.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:02:41.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:41.803 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:41.802+0000 7f4fae412780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:42.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:02:42.936 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3621197913,v1:127.0.0.1:6835/3621197913] [v2:127.0.0.1:6836/3621197913,v1:127.0.0.1:6837/3621197913] exists,up 9b93c283-2a9c-4d69-b72e-bb0a4f09d1e1 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:42.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:42.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-13T19:02:42.939 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:42.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c2d71f68-b176-44bc-9ac8-41d396c89cd1 2026-02-13T19:02:42.940 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 c2d71f68-b176-44bc-9ac8-41d396c89cd1 2026-02-13T19:02:42.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 c2d71f68-b176-44bc-9ac8-41d396c89cd1' 2026-02-13T19:02:42.940 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:42.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBSdY9p+SSYOBAAmEXuwXaKlZkKyNUe/8OeMA== 2026-02-13T19:02:42.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBSdY9p+SSYOBAAmEXuwXaKlZkKyNUe/8OeMA=="}' 2026-02-13T19:02:42.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c2d71f68-b176-44bc-9ac8-41d396c89cd1 -i td/test-erasure-code/5/new.json 2026-02-13T19:02:43.180 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:02:43.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-13T19:02:43.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBSdY9p+SSYOBAAmEXuwXaKlZkKyNUe/8OeMA== --osd-uuid c2d71f68-b176-44bc-9ac8-41d396c89cd1 2026-02-13T19:02:43.204 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.203+0000 7fd67fdd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:43.205 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.205+0000 7fd67fdd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:43.206 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.205+0000 7fd67fdd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:43.207 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.206+0000 7fd67fdd6780 -1 bdev(0x390ac00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:43.207 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.206+0000 7fd67fdd6780 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-13T19:02:43.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-13T19:02:43.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:43.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:02:43.535 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:02:43.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:43.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:43.872 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.871+0000 7fe80b443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:43.873 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.873+0000 7fe80b443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:43.874 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:43.873+0000 7fe80b443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:44.009 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:44.008+0000 7fe80b443780 -1 Falling back to public interface 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:44.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:44.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:44.085 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:44.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:44.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:02:44.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:44.373 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:44.372+0000 7fe80b443780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:02:45.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:45.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:45.309 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:45.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:45.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:45.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:02:45.535 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 28 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/232137692,v1:127.0.0.1:6843/232137692] [v2:127.0.0.1:6844/232137692,v1:127.0.0.1:6845/232137692] exists,up c2d71f68-b176-44bc-9ac8-41d396c89cd1 2026-02-13T19:02:45.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:45.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:45.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:45.541 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:45.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dc69736f-7105-4870-a1d7-0a5abe94dc0c 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 dc69736f-7105-4870-a1d7-0a5abe94dc0c' 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 dc69736f-7105-4870-a1d7-0a5abe94dc0c 2026-02-13T19:02:45.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:45.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBVdY9pcFCyIBAAP0U95leNwgpld2idC4iNoA== 2026-02-13T19:02:45.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBVdY9pcFCyIBAAP0U95leNwgpld2idC4iNoA=="}' 2026-02-13T19:02:45.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dc69736f-7105-4870-a1d7-0a5abe94dc0c -i td/test-erasure-code/6/new.json 2026-02-13T19:02:45.778 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:02:45.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-13T19:02:45.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBVdY9pcFCyIBAAP0U95leNwgpld2idC4iNoA== --osd-uuid dc69736f-7105-4870-a1d7-0a5abe94dc0c 2026-02-13T19:02:45.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:45.800+0000 7f952d03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:45.803 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:45.802+0000 7f952d03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:45.803 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:45.803+0000 7f952d03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:45.804 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:45.803+0000 7f952d03f780 -1 bdev(0x3ce8c00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:45.804 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:45.803+0000 7f952d03f780 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-13T19:02:46.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-13T19:02:46.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:46.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:02:46.136 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:02:46.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:46.453 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:02:46.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:02:46.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:46.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:46.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:46.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:46.468 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:46.467+0000 7fe7eec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:46.469 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:46.468+0000 7fe7eec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:46.470 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:46.469+0000 7fe7eec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:46.606 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:46.605+0000 7fe7eec43780 -1 Falling back to public interface 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:46.681 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:46.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:46.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:02:46.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:46.969 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:46.968+0000 7fe7eec43780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:02:47.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:47.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:47.908 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:47.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:47.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:47.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:02:48.137 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3559441486,v1:127.0.0.1:6851/3559441486] [v2:127.0.0.1:6852/3559441486,v1:127.0.0.1:6853/3559441486] exists,up dc69736f-7105-4870-a1d7-0a5abe94dc0c 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-13T19:02:48.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-13T19:02:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-13T19:02:48.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5abc5fb5-fe60-426d-a86b-0364d92c787d 2026-02-13T19:02:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 5abc5fb5-fe60-426d-a86b-0364d92c787d' 2026-02-13T19:02:48.144 INFO:tasks.workunit.client.0.trial034.stdout:add osd7 5abc5fb5-fe60-426d-a86b-0364d92c787d 2026-02-13T19:02:48.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBYdY9pHrr0CBAAs923t4haf1K148Bis98aRA== 2026-02-13T19:02:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBYdY9pHrr0CBAAs923t4haf1K148Bis98aRA=="}' 2026-02-13T19:02:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5abc5fb5-fe60-426d-a86b-0364d92c787d -i td/test-erasure-code/7/new.json 2026-02-13T19:02:48.384 INFO:tasks.workunit.client.0.trial034.stdout:7 2026-02-13T19:02:48.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-13T19:02:48.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBYdY9pHrr0CBAAs923t4haf1K148Bis98aRA== --osd-uuid 5abc5fb5-fe60-426d-a86b-0364d92c787d 2026-02-13T19:02:48.408 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:48.407+0000 7f5a72268780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:48.410 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:48.409+0000 7f5a72268780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:48.411 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:48.410+0000 7f5a72268780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:48.411 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:48.410+0000 7f5a72268780 -1 bdev(0x3b2ec00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:48.412 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:48.410+0000 7f5a72268780 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-13T19:02:48.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-13T19:02:48.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:48.746 INFO:tasks.workunit.client.0.trial034.stdout:adding osd7 key to auth repository 2026-02-13T19:02:48.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-13T19:02:48.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:49.064 INFO:tasks.workunit.client.0.trial034.stdout:start osd.7 2026-02-13T19:02:49.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-13T19:02:49.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:49.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:49.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:49.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:49.078 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:49.077+0000 7f4a55037780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:49.080 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:49.079+0000 7f4a55037780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:49.080 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:49.080+0000 7f4a55037780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:49.224 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:49.223+0000 7f4a55037780 -1 Falling back to public interface 2026-02-13T19:02:49.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-13T19:02:49.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:49.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-13T19:02:49.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:49.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:49.288 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:49.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:49.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:49.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:49.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-13T19:02:49.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:49.595 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:49.594+0000 7f4a55037780 -1 osd.7 0 log_to_monitors true 2026-02-13T19:02:50.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:50.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:50.515 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:50.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:50.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:50.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-13T19:02:50.737 INFO:tasks.workunit.client.0.trial034.stdout:osd.7 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/2362762507,v1:127.0.0.1:6859/2362762507] [v2:127.0.0.1:6860/2362762507,v1:127.0.0.1:6861/2362762507] exists,up 5abc5fb5-fe60-426d-a86b-0364d92c787d 2026-02-13T19:02:50.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:50.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:50.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:50.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:50.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:50.740 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-13T19:02:50.742 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:50.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f2fba241-a871-4722-86aa-d9995ae21870 2026-02-13T19:02:50.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 f2fba241-a871-4722-86aa-d9995ae21870' 2026-02-13T19:02:50.742 INFO:tasks.workunit.client.0.trial034.stdout:add osd8 f2fba241-a871-4722-86aa-d9995ae21870 2026-02-13T19:02:50.742 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:50.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBadY9pZD61LBAAAETUM1GlrL08/kCrCgaGdg== 2026-02-13T19:02:50.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBadY9pZD61LBAAAETUM1GlrL08/kCrCgaGdg=="}' 2026-02-13T19:02:50.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f2fba241-a871-4722-86aa-d9995ae21870 -i td/test-erasure-code/8/new.json 2026-02-13T19:02:50.982 INFO:tasks.workunit.client.0.trial034.stdout:8 2026-02-13T19:02:50.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-13T19:02:50.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBadY9pZD61LBAAAETUM1GlrL08/kCrCgaGdg== --osd-uuid f2fba241-a871-4722-86aa-d9995ae21870 2026-02-13T19:02:51.008 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.007+0000 7f6056612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.010 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.009+0000 7f6056612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.011 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.010+0000 7f6056612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.011 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.010+0000 7f6056612780 -1 bdev(0x3ebec00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:51.011 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.010+0000 7f6056612780 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-13T19:02:51.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-13T19:02:51.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:51.347 INFO:tasks.workunit.client.0.trial034.stdout:adding osd8 key to auth repository 2026-02-13T19:02:51.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-13T19:02:51.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:51.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-13T19:02:51.663 INFO:tasks.workunit.client.0.trial034.stdout:start osd.8 2026-02-13T19:02:51.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:51.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:51.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:51.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:51.677 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.676+0000 7f589ba57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.679 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.678+0000 7f589ba57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.679 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.679+0000 7f589ba57780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:51.826 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:51.824+0000 7f589ba57780 -1 Falling back to public interface 2026-02-13T19:02:51.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-13T19:02:51.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:51.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-13T19:02:51.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:51.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:51.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:51.892 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:51.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:51.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:51.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-13T19:02:52.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:52.125 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:52.124+0000 7f589ba57780 -1 osd.8 0 log_to_monitors true 2026-02-13T19:02:53.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:53.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:53.117 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:53.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:53.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:53.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-13T19:02:53.348 INFO:tasks.workunit.client.0.trial034.stdout:osd.8 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1868824566,v1:127.0.0.1:6867/1868824566] [v2:127.0.0.1:6868/1868824566,v1:127.0.0.1:6869/1868824566] exists,up f2fba241-a871-4722-86aa-d9995ae21870 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:53.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:53.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:53.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:53.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-13T19:02:53.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:53.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=778d9f16-96a3-415f-80ee-4531c69f5f3c 2026-02-13T19:02:53.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 778d9f16-96a3-415f-80ee-4531c69f5f3c' 2026-02-13T19:02:53.356 INFO:tasks.workunit.client.0.trial034.stdout:add osd9 778d9f16-96a3-415f-80ee-4531c69f5f3c 2026-02-13T19:02:53.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:53.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBddY9pLdV+FRAAvvuB96+zNujJbWX5w9275A== 2026-02-13T19:02:53.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBddY9pLdV+FRAAvvuB96+zNujJbWX5w9275A=="}' 2026-02-13T19:02:53.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 778d9f16-96a3-415f-80ee-4531c69f5f3c -i td/test-erasure-code/9/new.json 2026-02-13T19:02:53.591 INFO:tasks.workunit.client.0.trial034.stdout:9 2026-02-13T19:02:53.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-13T19:02:53.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBddY9pLdV+FRAAvvuB96+zNujJbWX5w9275A== --osd-uuid 778d9f16-96a3-415f-80ee-4531c69f5f3c 2026-02-13T19:02:53.615 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:53.614+0000 7fc93ce0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:53.616 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:53.616+0000 7fc93ce0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:53.617 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:53.617+0000 7fc93ce0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:53.618 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:53.617+0000 7fc93ce0f780 -1 bdev(0x2cc2c00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:53.618 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:53.617+0000 7fc93ce0f780 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-13T19:02:53.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-13T19:02:53.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:53.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-13T19:02:53.945 INFO:tasks.workunit.client.0.trial034.stdout:adding osd9 key to auth repository 2026-02-13T19:02:53.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stdout:start osd.9 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:54.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:54.294 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:54.293+0000 7f458a43f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:54.295 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:54.294+0000 7f458a43f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:54.296 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:54.295+0000 7f458a43f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:54.428 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:54.427+0000 7f458a43f780 -1 Falling back to public interface 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:54.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:54.508 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:54.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:54.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:54.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-13T19:02:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:54.781 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:54.781+0000 7f458a43f780 -1 osd.9 0 log_to_monitors true 2026-02-13T19:02:55.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:55.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:55.733 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:55.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:55.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:55.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-13T19:02:55.958 INFO:tasks.workunit.client.0.trial034.stdout:osd.9 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/1039375132,v1:127.0.0.1:6875/1039375132] [v2:127.0.0.1:6876/1039375132,v1:127.0.0.1:6877/1039375132] exists,up 778d9f16-96a3-415f-80ee-4531c69f5f3c 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:02:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:02:55.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-13T19:02:55.961 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:02:55.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=53a4d2b5-897d-4beb-a31a-9fee472aec22 2026-02-13T19:02:55.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 53a4d2b5-897d-4beb-a31a-9fee472aec22' 2026-02-13T19:02:55.962 INFO:tasks.workunit.client.0.trial034.stdout:add osd10 53a4d2b5-897d-4beb-a31a-9fee472aec22 2026-02-13T19:02:55.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:02:55.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBfdY9pC0blORAAZ9X30XJicxllQmgGv7sfYQ== 2026-02-13T19:02:55.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBfdY9pC0blORAAZ9X30XJicxllQmgGv7sfYQ=="}' 2026-02-13T19:02:55.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 53a4d2b5-897d-4beb-a31a-9fee472aec22 -i td/test-erasure-code/10/new.json 2026-02-13T19:02:56.204 INFO:tasks.workunit.client.0.trial034.stdout:10 2026-02-13T19:02:56.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-13T19:02:56.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBfdY9pC0blORAAZ9X30XJicxllQmgGv7sfYQ== --osd-uuid 53a4d2b5-897d-4beb-a31a-9fee472aec22 2026-02-13T19:02:56.226 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.225+0000 7f18bc20a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:56.227 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.227+0000 7f18bc20a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:56.228 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.227+0000 7f18bc20a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:56.229 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.228+0000 7f18bc20a780 -1 bdev(0x3ba6c00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-13T19:02:56.229 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.228+0000 7f18bc20a780 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-13T19:02:56.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-13T19:02:56.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:02:56.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-13T19:02:56.563 INFO:tasks.workunit.client.0.trial034.stdout:adding osd10 key to auth repository 2026-02-13T19:02:56.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:02:56.881 INFO:tasks.workunit.client.0.trial034.stdout:start osd.10 2026-02-13T19:02:56.881 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-13T19:02:56.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:02:56.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:02:56.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:02:56.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:02:56.896 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.896+0000 7f335f851780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:56.898 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.897+0000 7f335f851780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:56.899 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:56.898+0000 7f335f851780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:02:57.019 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:57.018+0000 7f335f851780 -1 Falling back to public interface 2026-02-13T19:02:57.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-13T19:02:57.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:02:57.110 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:02:57.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:57.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-13T19:02:57.320 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:02:57.319+0000 7f335f851780 -1 osd.10 0 log_to_monitors true 2026-02-13T19:02:57.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:02:58.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:02:58.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:02:58.336 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:02:58.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:02:58.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:02:58.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-13T19:02:58.561 INFO:tasks.workunit.client.0.trial034.stdout:osd.10 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/3721132792,v1:127.0.0.1:6883/3721132792] [v2:127.0.0.1:6884/3721132792,v1:127.0.0.1:6885/3721132792] exists,up 53a4d2b5-897d-4beb-a31a-9fee472aec22 2026-02-13T19:02:58.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:02:58.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:02:58.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:02:58.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-13T19:02:58.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-13T19:02:58.781 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' does not exist 2026-02-13T19:02:58.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-13T19:02:58.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:02:59.062 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:02:59.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:03:00.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-13T19:03:00.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-13T19:03:00.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:00.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:00.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:00.351 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:00.351 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:00.352 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:00.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:00.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:00.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:00.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:00.660 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:00.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-13T19:03:00.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-13T19:03:00.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488' 2026-02-13T19:03:00.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.726 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:00.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672968 2026-02-13T19:03:00.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672968 2026-02-13T19:03:00.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968' 2026-02-13T19:03:00.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:00.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542151 2026-02-13T19:03:00.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542151 2026-02-13T19:03:00.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151' 2026-02-13T19:03:00.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:00.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378631 2026-02-13T19:03:00.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378631 2026-02-13T19:03:00.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631' 2026-02-13T19:03:00.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.916 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:00.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247814 2026-02-13T19:03:00.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247814 2026-02-13T19:03:00.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814' 2026-02-13T19:03:00.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:00.980 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:01.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084295 2026-02-13T19:03:01.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084295 2026-02-13T19:03:01.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295' 2026-02-13T19:03:01.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:01.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:01.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920774 2026-02-13T19:03:01.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920774 2026-02-13T19:03:01.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295 6-141733920774' 2026-02-13T19:03:01.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:01.107 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:01.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757253 2026-02-13T19:03:01.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757253 2026-02-13T19:03:01.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295 6-141733920774 7-163208757253' 2026-02-13T19:03:01.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:01.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:01.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-13T19:03:01.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-13T19:03:01.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295 6-141733920774 7-163208757253 8-184683593733' 2026-02-13T19:03:01.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:01.234 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:01.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462916 2026-02-13T19:03:01.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462916 2026-02-13T19:03:01.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295 6-141733920774 7-163208757253 8-184683593733 9-201863462916' 2026-02-13T19:03:01.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:01.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:01.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-13T19:03:01.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-13T19:03:01.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672968 2-60129542151 3-81604378631 4-98784247814 5-120259084295 6-141733920774 7-163208757253 8-184683593733 9-201863462916 10-223338299395' 2026-02-13T19:03:01.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:01.362 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836488 2026-02-13T19:03:01.362 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:01.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:01.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-13T19:03:01.364 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:01.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836488 2026-02-13T19:03:01.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-13T19:03:01.365 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836488 2026-02-13T19:03:01.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:01.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-13T19:03:01.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:03:02.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:03:02.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:02.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-13T19:03:02.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:02.822 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672968 2026-02-13T19:03:02.822 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:02.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:02.824 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672968 2026-02-13T19:03:02.824 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:02.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672968 2026-02-13T19:03:02.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-13T19:03:02.825 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949672968 2026-02-13T19:03:02.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:03.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-13T19:03:03.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:03.049 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542151 2026-02-13T19:03:03.049 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:03.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:03.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542151 2026-02-13T19:03:03.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:03.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542151 2026-02-13T19:03:03.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-13T19:03:03.052 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542151 2026-02-13T19:03:03.052 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:03.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-13T19:03:03.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:03.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378631 2026-02-13T19:03:03.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:03.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:03.283 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378631 2026-02-13T19:03:03.283 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:03.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378631 2026-02-13T19:03:03.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378631' 2026-02-13T19:03:03.285 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 81604378631 2026-02-13T19:03:03.285 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:03.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-13T19:03:03.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:03.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247814 2026-02-13T19:03:03.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:03.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:03.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247814 2026-02-13T19:03:03.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:03.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247814 2026-02-13T19:03:03.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247814' 2026-02-13T19:03:03.517 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 98784247814 2026-02-13T19:03:03.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:03.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-13T19:03:03.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:03.745 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084295 2026-02-13T19:03:03.745 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:03.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:03.746 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084295 2026-02-13T19:03:03.747 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:03.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084295 2026-02-13T19:03:03.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084295' 2026-02-13T19:03:03.748 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 120259084295 2026-02-13T19:03:03.749 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:03.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084295 -lt 120259084295 2026-02-13T19:03:03.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:03.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920774 2026-02-13T19:03:03.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:03.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:03.977 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920774 2026-02-13T19:03:03.977 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:03.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920774 2026-02-13T19:03:03.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920774' 2026-02-13T19:03:03.979 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 141733920774 2026-02-13T19:03:03.979 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:04.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-13T19:03:04.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:04.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757253 2026-02-13T19:03:04.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:04.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:04.207 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757253 2026-02-13T19:03:04.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:04.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757253 2026-02-13T19:03:04.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757253' 2026-02-13T19:03:04.209 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 163208757253 2026-02-13T19:03:04.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:04.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757253 -lt 163208757253 2026-02-13T19:03:04.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:04.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593733 2026-02-13T19:03:04.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:04.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:04.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593733 2026-02-13T19:03:04.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:04.437 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593733 2026-02-13T19:03:04.437 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593733' 2026-02-13T19:03:04.438 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 184683593733 2026-02-13T19:03:04.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:04.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593734 -lt 184683593733 2026-02-13T19:03:04.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:04.663 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462916 2026-02-13T19:03:04.664 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:04.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:04.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462916 2026-02-13T19:03:04.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:04.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462916 2026-02-13T19:03:04.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462916' 2026-02-13T19:03:04.666 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.9 seq 201863462916 2026-02-13T19:03:04.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:04.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462916 -lt 201863462916 2026-02-13T19:03:04.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:04.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299395 2026-02-13T19:03:04.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:04.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:04.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299395 2026-02-13T19:03:04.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:04.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299395 2026-02-13T19:03:04.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299395' 2026-02-13T19:03:04.894 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.10 seq 223338299395 2026-02-13T19:03:04.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:05.121 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299396 -lt 223338299395 2026-02-13T19:03:05.122 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:05.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:05.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:05.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-13T19:03:05.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:05.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:05.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:05.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:05.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:05.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:05.435 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:05.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:03:05.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:05.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:05.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:05.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.66758/ceph-osd.0.asok 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-13T19:03:05.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.66758/ceph-osd.0.asok log flush 2026-02-13T19:03:06.011 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:03:06.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-13T19:03:06.017 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:02:31.503+0000 7f1ddf03d780 0 load: jerasure load: lrc 2026-02-13T19:03:06.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-13T19:03:06.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-13T19:03:06.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-13T19:03:06.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-13T19:03:06.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-13T19:03:06.683 INFO:tasks.workunit.client.0.trial034.stderr:pool 'ecpool' already exists 2026-02-13T19:03:06.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:03:07.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:03:07.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:07.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:07.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:07.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:07.778 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:08.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:08.022 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:08.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:08.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:08.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-13T19:03:08.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-13T19:03:08.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492' 2026-02-13T19:03:08.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.084 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:08.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672971 2026-02-13T19:03:08.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672971 2026-02-13T19:03:08.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2026-02-13T19:03:08.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:08.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542155 2026-02-13T19:03:08.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542155 2026-02-13T19:03:08.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155' 2026-02-13T19:03:08.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.214 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:08.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378634 2026-02-13T19:03:08.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378634 2026-02-13T19:03:08.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634' 2026-02-13T19:03:08.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.281 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:08.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247818 2026-02-13T19:03:08.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247818 2026-02-13T19:03:08.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818' 2026-02-13T19:03:08.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:08.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084298 2026-02-13T19:03:08.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084298 2026-02-13T19:03:08.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298' 2026-02-13T19:03:08.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:08.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920778 2026-02-13T19:03:08.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920778 2026-02-13T19:03:08.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298 6-141733920778' 2026-02-13T19:03:08.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:08.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757256 2026-02-13T19:03:08.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757256 2026-02-13T19:03:08.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298 6-141733920778 7-163208757256' 2026-02-13T19:03:08.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.542 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:08.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593737 2026-02-13T19:03:08.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593737 2026-02-13T19:03:08.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298 6-141733920778 7-163208757256 8-184683593737' 2026-02-13T19:03:08.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:08.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-13T19:03:08.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-13T19:03:08.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298 6-141733920778 7-163208757256 8-184683593737 9-201863462919' 2026-02-13T19:03:08.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:08.672 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299399 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299399 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542155 3-81604378634 4-98784247818 5-120259084298 6-141733920778 7-163208757256 8-184683593737 9-201863462919 10-223338299399' 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836492 2026-02-13T19:03:08.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:08.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-13T19:03:08.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:08.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836492 2026-02-13T19:03:08.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-13T19:03:08.738 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836492 2026-02-13T19:03:08.738 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:08.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-13T19:03:08.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:08.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672971 2026-02-13T19:03:08.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:08.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:08.967 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672971 2026-02-13T19:03:08.967 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:08.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672971 2026-02-13T19:03:08.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-13T19:03:08.969 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949672971 2026-02-13T19:03:08.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:09.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-13T19:03:09.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:09.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542155 2026-02-13T19:03:09.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:09.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:09.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542155 2026-02-13T19:03:09.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:09.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542155 2026-02-13T19:03:09.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-13T19:03:09.208 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542155 2026-02-13T19:03:09.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:09.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-13T19:03:09.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:09.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378634 2026-02-13T19:03:09.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:09.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:09.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378634 2026-02-13T19:03:09.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:09.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378634 2026-02-13T19:03:09.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378634' 2026-02-13T19:03:09.440 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 81604378634 2026-02-13T19:03:09.440 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:09.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378634 -lt 81604378634 2026-02-13T19:03:09.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:09.670 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247818 2026-02-13T19:03:09.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:09.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:09.672 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247818 2026-02-13T19:03:09.672 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:09.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247818 2026-02-13T19:03:09.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247818' 2026-02-13T19:03:09.674 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 98784247818 2026-02-13T19:03:09.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:09.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247818 -lt 98784247818 2026-02-13T19:03:09.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:09.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084298 2026-02-13T19:03:09.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:09.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:09.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084298 2026-02-13T19:03:09.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:09.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084298 2026-02-13T19:03:09.901 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 120259084298 2026-02-13T19:03:09.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084298' 2026-02-13T19:03:09.902 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:10.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084298 -lt 120259084298 2026-02-13T19:03:10.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:10.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920778 2026-02-13T19:03:10.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:10.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:10.131 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920778 2026-02-13T19:03:10.131 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:10.132 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920778 2026-02-13T19:03:10.132 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920778' 2026-02-13T19:03:10.132 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 141733920778 2026-02-13T19:03:10.132 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:10.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920778 -lt 141733920778 2026-02-13T19:03:10.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:10.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757256 2026-02-13T19:03:10.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:10.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:10.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757256 2026-02-13T19:03:10.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:10.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757256 2026-02-13T19:03:10.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757256' 2026-02-13T19:03:10.369 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 163208757256 2026-02-13T19:03:10.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:10.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757256 -lt 163208757256 2026-02-13T19:03:10.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:10.598 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593737 2026-02-13T19:03:10.599 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:10.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:10.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593737 2026-02-13T19:03:10.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:10.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593737 2026-02-13T19:03:10.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593737' 2026-02-13T19:03:10.601 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 184683593737 2026-02-13T19:03:10.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:10.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593737 -lt 184683593737 2026-02-13T19:03:10.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:10.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462919 2026-02-13T19:03:10.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:10.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:10.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462919 2026-02-13T19:03:10.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:10.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462919 2026-02-13T19:03:10.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462919' 2026-02-13T19:03:10.831 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.9 seq 201863462919 2026-02-13T19:03:10.831 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:11.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-13T19:03:11.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:11.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299399 2026-02-13T19:03:11.060 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:11.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:11.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299399 2026-02-13T19:03:11.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:11.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299399 2026-02-13T19:03:11.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299399' 2026-02-13T19:03:11.062 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.10 seq 223338299399 2026-02-13T19:03:11.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:11.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299399 -lt 223338299399 2026-02-13T19:03:11.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:11.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:11.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:11.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:11.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:11.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:11.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-13T19:03:11.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:11.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:11.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:12.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-13T19:03:12.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:12.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:12.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-13T19:03:12.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_chunk_mapping 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_put_get_isa 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_put_get_jerasure 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_put_get_lrc_advanced 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_put_get_lrc_kml 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_put_get_shec' 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-13T19:03:12.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-13T19:03:12.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-13T19:03:12.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-13T19:03:12.161 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-13T19:03:12.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-13T19:03:12.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-13T19:03:12.169 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-13T19:03:12.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-13T19:03:12.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-13T19:03:12.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-13T19:03:12.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-13T19:03:12.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-13T19:03:12.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-13T19:03:12.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-13T19:03:12.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-13T19:03:12.411 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-13T19:03:12.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-13T19:03:12.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-13T19:03:12.412 INFO:tasks.workunit.client.0.trial034.stderr:2+0 records in 2026-02-13T19:03:12.412 INFO:tasks.workunit.client.0.trial034.stderr:2+0 records out 2026-02-13T19:03:12.412 INFO:tasks.workunit.client.0.trial034.stderr:16382 bytes (16 kB, 16 KiB) copied, 7.0712e-05 s, 232 MB/s 2026-02-13T19:03:12.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-13T19:03:12.427 INFO:tasks.workunit.client.0.trial034.stderr:INFO: op_size has been rounded to 8192 2026-02-13T19:03:12.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:03:12.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:03:12.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-13T19:03:12.433 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-13T19:03:12.434 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-13T19:03:12.441 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-13T19:03:12.441 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-13T19:03:12.441 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-13T19:03:12.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-13T19:03:12.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-13T19:03:12.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-13T19:03:12.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-13T19:03:12.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-13T19:03:12.451 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-13T19:03:12.451 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-13T19:03:12.458 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-13T19:03:12.458 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-13T19:03:12.458 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-13T19:03:12.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-13T19:03:12.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-13T19:03:12.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-13T19:03:12.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-13T19:03:12.468 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-13T19:03:12.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-13T19:03:12.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-13T19:03:12.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-13T19:03:12.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-13T19:03:12.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-13T19:03:12.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-13T19:03:12.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-13T19:03:12.505 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-13T19:03:12.505 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:12.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:0' 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-13T19:03:12.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-13T19:03:12.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-13T19:03:12.783 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:12.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:12.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-13T19:03:12.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-13T19:03:12.835 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:12.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:12.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-13T19:03:12.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-13T19:03:12.885 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:12.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:12.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-13T19:03:12.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-13T19:03:12.937 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:12.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:12.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-13T19:03:12.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:03:12.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-13T19:03:12.940 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-13T19:03:12.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-13T19:03:12.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-13T19:03:12.941 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-13T19:03:12.941 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:13.170 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:0' 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-13T19:03:13.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:03:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:03:13.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:03:13.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:03:13.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-13T19:03:13.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-13T19:03:13.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-13T19:03:13.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-13T19:03:13.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:03:13.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:03:13.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:start osd.3 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:03:13.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-13T19:03:13.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:03:13.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:03:13.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:03:13.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:03:13.600 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:13.599+0000 7f972782b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:13.601 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:13.601+0000 7f972782b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:13.602 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:13.601+0000 7f972782b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:13.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:13.800+0000 7f972782b780 -1 Falling back to public interface 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:03:13.818 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:13.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:13.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:03:14.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:03:14.157 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:14.156+0000 7f972782b780 -1 osd.3 70 log_to_monitors true 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [19,70) [v2:127.0.0.1:6826/3538470622,v1:127.0.0.1:6827/3538470622] [v2:127.0.0.1:6828/3538470622,v1:127.0.0.1:6829/3538470622] exists,up 364600fc-dc7e-4e02-9179-dbb88e787609 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:15.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:15.268 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:15.268 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:15.268 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:15.268 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:15.268 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:15.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:15.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:15.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:15.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:15.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:15.349 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:15.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:15.575 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:15.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:15.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:15.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-13T19:03:15.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-13T19:03:15.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495' 2026-02-13T19:03:15.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:15.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672975 2026-02-13T19:03:15.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672975 2026-02-13T19:03:15.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975' 2026-02-13T19:03:15.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.701 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:15.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542158 2026-02-13T19:03:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542158 2026-02-13T19:03:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158' 2026-02-13T19:03:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579907 2026-02-13T19:03:15.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579907 2026-02-13T19:03:15.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907' 2026-02-13T19:03:15.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.830 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:15.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247821 2026-02-13T19:03:15.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247821 2026-02-13T19:03:15.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821' 2026-02-13T19:03:15.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:15.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084302 2026-02-13T19:03:15.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084302 2026-02-13T19:03:15.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302' 2026-02-13T19:03:15.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:15.956 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:16.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920781 2026-02-13T19:03:16.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920781 2026-02-13T19:03:16.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302 6-141733920781' 2026-02-13T19:03:16.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:16.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:16.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757260 2026-02-13T19:03:16.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757260 2026-02-13T19:03:16.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302 6-141733920781 7-163208757260' 2026-02-13T19:03:16.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:16.085 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:16.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593740 2026-02-13T19:03:16.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593740 2026-02-13T19:03:16.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302 6-141733920781 7-163208757260 8-184683593740' 2026-02-13T19:03:16.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:16.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:16.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462923 2026-02-13T19:03:16.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462923 2026-02-13T19:03:16.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302 6-141733920781 7-163208757260 8-184683593740 9-201863462923' 2026-02-13T19:03:16.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:16.214 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:16.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299402 2026-02-13T19:03:16.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299402 2026-02-13T19:03:16.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-42949672975 2-60129542158 3-317827579907 4-98784247821 5-120259084302 6-141733920781 7-163208757260 8-184683593740 9-201863462923 10-223338299402' 2026-02-13T19:03:16.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:16.277 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836495 2026-02-13T19:03:16.277 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:16.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:16.278 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-13T19:03:16.278 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:16.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836495 2026-02-13T19:03:16.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-13T19:03:16.279 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 21474836495 2026-02-13T19:03:16.279 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:16.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-13T19:03:16.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:03:17.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:03:17.509 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:17.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-13T19:03:17.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:17.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672975 2026-02-13T19:03:17.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:17.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:17.738 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672975 2026-02-13T19:03:17.738 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:17.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672975 2026-02-13T19:03:17.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-13T19:03:17.739 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 42949672975 2026-02-13T19:03:17.740 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:17.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-13T19:03:17.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:17.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542158 2026-02-13T19:03:17.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:17.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:17.963 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542158 2026-02-13T19:03:17.963 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:17.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542158 2026-02-13T19:03:17.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-13T19:03:17.965 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 60129542158 2026-02-13T19:03:17.965 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:18.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-13T19:03:18.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:18.200 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579907 2026-02-13T19:03:18.200 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:18.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:18.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579907 2026-02-13T19:03:18.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:18.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579907 2026-02-13T19:03:18.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579907' 2026-02-13T19:03:18.202 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 317827579907 2026-02-13T19:03:18.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:18.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-13T19:03:18.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:18.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247821 2026-02-13T19:03:18.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:18.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:18.430 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247821 2026-02-13T19:03:18.430 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:18.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247821 2026-02-13T19:03:18.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247821' 2026-02-13T19:03:18.431 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.4 seq 98784247821 2026-02-13T19:03:18.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:18.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247822 -lt 98784247821 2026-02-13T19:03:18.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:18.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-120259084302 2026-02-13T19:03:18.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:18.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:18.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084302 2026-02-13T19:03:18.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:18.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=120259084302 2026-02-13T19:03:18.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 120259084302' 2026-02-13T19:03:18.662 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.5 seq 120259084302 2026-02-13T19:03:18.663 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:18.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 120259084302 -lt 120259084302 2026-02-13T19:03:18.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:18.897 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920781 2026-02-13T19:03:18.898 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:18.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:18.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920781 2026-02-13T19:03:18.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:18.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920781 2026-02-13T19:03:18.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920781' 2026-02-13T19:03:18.900 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.6 seq 141733920781 2026-02-13T19:03:18.900 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:19.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920782 -lt 141733920781 2026-02-13T19:03:19.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:19.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757260 2026-02-13T19:03:19.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:19.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:19.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757260 2026-02-13T19:03:19.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757260 2026-02-13T19:03:19.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757260' 2026-02-13T19:03:19.131 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.7 seq 163208757260 2026-02-13T19:03:19.131 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:19.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757260 -lt 163208757260 2026-02-13T19:03:19.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:19.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593740 2026-02-13T19:03:19.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:19.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:19.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593740 2026-02-13T19:03:19.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:19.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593740 2026-02-13T19:03:19.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593740' 2026-02-13T19:03:19.357 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.8 seq 184683593740 2026-02-13T19:03:19.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:19.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593741 -lt 184683593740 2026-02-13T19:03:19.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:19.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462923 2026-02-13T19:03:19.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:19.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:19.582 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462923 2026-02-13T19:03:19.582 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:19.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462923 2026-02-13T19:03:19.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462923' 2026-02-13T19:03:19.584 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.9 seq 201863462923 2026-02-13T19:03:19.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:19.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462923 -lt 201863462923 2026-02-13T19:03:19.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:19.816 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299402 2026-02-13T19:03:19.816 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:19.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:19.817 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299402 2026-02-13T19:03:19.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:19.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299402 2026-02-13T19:03:19.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299402' 2026-02-13T19:03:19.819 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.10 seq 223338299402 2026-02-13T19:03:19.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:20.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299403 -lt 223338299402 2026-02-13T19:03:20.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:20.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:20.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-13T19:03:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:20.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:20.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:20.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:20.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:20.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:20.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:20.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-13T19:03:20.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:20.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:20.585 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:20.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-13T19:03:20.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:20.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:20.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-13T19:03:20.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-13T19:03:20.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:03:20.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:03:20.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:03:20.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:03:21.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:03:21.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-13T19:03:21.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-13T19:03:21.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:03:21.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-13T19:03:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:03:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-13T19:03:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-13T19:03:21.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:03:21.092 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-13T19:03:21.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-13T19:03:21.094 INFO:tasks.workunit.client.0.trial034.stderr:start osd.5 2026-02-13T19:03:21.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:03:21.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-13T19:03:21.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-13T19:03:21.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:03:21.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:03:21.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:03:21.105 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:21.103+0000 7f279ba10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:21.107 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:21.105+0000 7f279ba10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:21.108 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:21.106+0000 7f279ba10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:21.236 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:21.234+0000 7f279ba10780 -1 Falling back to public interface 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:03:21.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:21.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:03:21.321 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:21.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:21.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:03:21.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:03:21.643 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:21.640+0000 7f279ba10780 -1 osd.5 77 log_to_monitors true 2026-02-13T19:03:22.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:03:22.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:22.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:03:22.548 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:22.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:22.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:03:22.781 INFO:tasks.workunit.client.0.trial034.stderr:osd.5 up in weight 1 up_from 80 up_thru 80 down_at 77 last_clean_interval [28,77) [v2:127.0.0.1:6842/2834807002,v1:127.0.0.1:6843/2834807002] [v2:127.0.0.1:6844/2834807002,v1:127.0.0.1:6845/2834807002] exists,up c2d71f68-b176-44bc-9ac8-41d396c89cd1 2026-02-13T19:03:22.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:03:22.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:03:22.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:22.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:22.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:22.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:22.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:22.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:22.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:22.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:23.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:23.089 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:23.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:23.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:23.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-13T19:03:23.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-13T19:03:23.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499' 2026-02-13T19:03:23.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:23.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672978 2026-02-13T19:03:23.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672978 2026-02-13T19:03:23.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978' 2026-02-13T19:03:23.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:23.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542162 2026-02-13T19:03:23.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542162 2026-02-13T19:03:23.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162' 2026-02-13T19:03:23.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.283 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:23.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579910 2026-02-13T19:03:23.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579910 2026-02-13T19:03:23.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910' 2026-02-13T19:03:23.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:23.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247825 2026-02-13T19:03:23.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247825 2026-02-13T19:03:23.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825' 2026-02-13T19:03:23.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:23.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383683 2026-02-13T19:03:23.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383683 2026-02-13T19:03:23.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683' 2026-02-13T19:03:23.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.478 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:23.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920785 2026-02-13T19:03:23.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920785 2026-02-13T19:03:23.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683 6-141733920785' 2026-02-13T19:03:23.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.545 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:23.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757263 2026-02-13T19:03:23.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757263 2026-02-13T19:03:23.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683 6-141733920785 7-163208757263' 2026-02-13T19:03:23.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:23.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593744 2026-02-13T19:03:23.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593744 2026-02-13T19:03:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683 6-141733920785 7-163208757263 8-184683593744' 2026-02-13T19:03:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:23.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462926 2026-02-13T19:03:23.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462926 2026-02-13T19:03:23.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683 6-141733920785 7-163208757263 8-184683593744 9-201863462926' 2026-02-13T19:03:23.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:23.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:23.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299406 2026-02-13T19:03:23.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299406 2026-02-13T19:03:23.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-42949672978 2-60129542162 3-317827579910 4-98784247825 5-343597383683 6-141733920785 7-163208757263 8-184683593744 9-201863462926 10-223338299406' 2026-02-13T19:03:23.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:23.799 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836499 2026-02-13T19:03:23.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:23.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:23.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-13T19:03:23.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:23.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836499 2026-02-13T19:03:23.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-13T19:03:23.801 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 21474836499 2026-02-13T19:03:23.802 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:24.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836496 -lt 21474836499 2026-02-13T19:03:24.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:03:25.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:03:25.026 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:25.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-13T19:03:25.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:25.250 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672978 2026-02-13T19:03:25.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:25.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:25.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672978 2026-02-13T19:03:25.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:25.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672978 2026-02-13T19:03:25.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672978' 2026-02-13T19:03:25.253 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 42949672978 2026-02-13T19:03:25.253 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:25.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672978 -lt 42949672978 2026-02-13T19:03:25.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:25.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542162 2026-02-13T19:03:25.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:25.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:25.480 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542162 2026-02-13T19:03:25.480 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:25.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542162 2026-02-13T19:03:25.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542162' 2026-02-13T19:03:25.481 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 60129542162 2026-02-13T19:03:25.482 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:25.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542162 -lt 60129542162 2026-02-13T19:03:25.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:25.706 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579910 2026-02-13T19:03:25.706 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:25.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:25.707 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579910 2026-02-13T19:03:25.707 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:25.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579910 2026-02-13T19:03:25.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579910' 2026-02-13T19:03:25.708 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 317827579910 2026-02-13T19:03:25.708 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:25.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579911 -lt 317827579910 2026-02-13T19:03:25.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:25.935 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247825 2026-02-13T19:03:25.935 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:25.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:25.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247825 2026-02-13T19:03:25.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:25.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247825 2026-02-13T19:03:25.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247825' 2026-02-13T19:03:25.937 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.4 seq 98784247825 2026-02-13T19:03:25.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:26.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247825 -lt 98784247825 2026-02-13T19:03:26.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:26.165 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383683 2026-02-13T19:03:26.165 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:26.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:26.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383683 2026-02-13T19:03:26.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:26.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383683 2026-02-13T19:03:26.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383683' 2026-02-13T19:03:26.167 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.5 seq 343597383683 2026-02-13T19:03:26.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:26.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383683 -lt 343597383683 2026-02-13T19:03:26.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:26.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-141733920785 2026-02-13T19:03:26.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:26.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:26.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920785 2026-02-13T19:03:26.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:26.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920785 2026-02-13T19:03:26.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 141733920785' 2026-02-13T19:03:26.400 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.6 seq 141733920785 2026-02-13T19:03:26.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:26.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920785 -lt 141733920785 2026-02-13T19:03:26.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:26.626 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757263 2026-02-13T19:03:26.626 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:26.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:26.627 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757263 2026-02-13T19:03:26.627 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:26.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757263 2026-02-13T19:03:26.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757263' 2026-02-13T19:03:26.628 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.7 seq 163208757263 2026-02-13T19:03:26.628 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:26.856 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757264 -lt 163208757263 2026-02-13T19:03:26.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:26.857 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593744 2026-02-13T19:03:26.857 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:26.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:26.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593744 2026-02-13T19:03:26.859 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:26.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593744 2026-02-13T19:03:26.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593744' 2026-02-13T19:03:26.860 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.8 seq 184683593744 2026-02-13T19:03:26.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:27.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593744 -lt 184683593744 2026-02-13T19:03:27.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:27.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462926 2026-02-13T19:03:27.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:27.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:27.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462926 2026-02-13T19:03:27.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:27.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462926 2026-02-13T19:03:27.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462926' 2026-02-13T19:03:27.090 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.9 seq 201863462926 2026-02-13T19:03:27.091 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:27.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462927 -lt 201863462926 2026-02-13T19:03:27.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:27.314 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299406 2026-02-13T19:03:27.314 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:27.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:27.315 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299406 2026-02-13T19:03:27.315 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:27.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299406 2026-02-13T19:03:27.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299406' 2026-02-13T19:03:27.316 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.10 seq 223338299406 2026-02-13T19:03:27.317 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:27.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299406 -lt 223338299406 2026-02-13T19:03:27.540 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:27.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:27.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:27.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:27.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:28.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-13T19:03:28.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:28.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:28.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:28.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-13T19:03:28.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:28.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:28.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-13T19:03:28.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-13T19:03:28.917 INFO:tasks.workunit.client.0.trial034.stdout:crush-device-class= 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:crush-failure-domain=host 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:crush-num-failure-domains=0 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:crush-osds-per-failure-domain=0 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:crush-root=default 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:k=-1 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:l=-1 2026-02-13T19:03:28.918 INFO:tasks.workunit.client.0.trial034.stdout:layers=[ [ "cDD", "" ] ] 2026-02-13T19:03:28.919 INFO:tasks.workunit.client.0.trial034.stdout:m=-1 2026-02-13T19:03:28.919 INFO:tasks.workunit.client.0.trial034.stdout:mapping=_DD 2026-02-13T19:03:28.919 INFO:tasks.workunit.client.0.trial034.stdout:plugin=lrc 2026-02-13T19:03:28.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-13T19:03:28.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-13T19:03:29.269 INFO:tasks.workunit.client.0.trial034.stderr:pool 'remap-pool' already exists 2026-02-13T19:03:29.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:03:30.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-13T19:03:30.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-13T19:03:30.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-13T19:03:30.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-13T19:03:30.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-13T19:03:30.279 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-13T19:03:30.279 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-13T19:03:30.288 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-13T19:03:30.288 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-13T19:03:30.288 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-13T19:03:30.297 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-13T19:03:30.298 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-13T19:03:30.306 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-13T19:03:30.306 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-13T19:03:30.307 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-13T19:03:30.316 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-13T19:03:30.316 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-13T19:03:30.316 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-13T19:03:30.317 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-13T19:03:30.317 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-13T19:03:30.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-13T19:03:30.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-13T19:03:30.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-13T19:03:30.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-13T19:03:30.350 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-13T19:03:30.350 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-13T19:03:30.351 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-13T19:03:30.351 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-13T19:03:30.351 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:30.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-13T19:03:30.581 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:9' 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-13T19:03:30.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-13T19:03:30.632 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:30.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:30.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-13T19:03:30.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-13T19:03:30.684 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:30.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:30.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-13T19:03:30.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-13T19:03:30.736 INFO:tasks.workunit.client.0.trial034.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-13T19:03:30.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-13T19:03:30.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-13T19:03:30.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:03:30.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-13T19:03:30.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-13T19:03:30.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-13T19:03:30.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-13T19:03:30.740 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-13T19:03:30.740 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:30.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-13T19:03:30.968 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:30.968 INFO:tasks.workunit.client.0.trial034.stderr:9' 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-13T19:03:30.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:03:30.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:03:31.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-13T19:03:31.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:31.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-13T19:03:31.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-13T19:03:31.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:03:31.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:03:31.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:start osd.6 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-13T19:03:31.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-13T19:03:31.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:03:31.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:03:31.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:03:31.376 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:31.374+0000 7f0ea9e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:31.377 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:31.375+0000 7f0ea9e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:31.378 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:31.376+0000 7f0ea9e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:31.562 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:31.560+0000 7f0ea9e2e780 -1 Falling back to public interface 2026-02-13T19:03:31.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-13T19:03:31.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:03:31.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:03:31.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:03:31.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:03:31.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:31.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:03:31.596 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:31.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:31.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:03:31.832 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:03:31.926 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:31.924+0000 7f0ea9e2e780 -1 osd.6 89 log_to_monitors true 2026-02-13T19:03:32.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:03:32.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:32.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:03:32.834 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:32.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:32.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:03:33.060 INFO:tasks.workunit.client.0.trial034.stderr:osd.6 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [33,89) [v2:127.0.0.1:6850/2852371637,v1:127.0.0.1:6851/2852371637] [v2:127.0.0.1:6852/2852371637,v1:127.0.0.1:6853/2852371637] exists,up dc69736f-7105-4870-a1d7-0a5abe94dc0c 2026-02-13T19:03:33.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:03:33.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:03:33.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:33.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:33.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:33.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:33.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:33.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:33.141 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:33.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:33.365 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:33.365 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:33.365 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:33.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-13T19:03:33.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-13T19:03:33.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503' 2026-02-13T19:03:33.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:33.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672982 2026-02-13T19:03:33.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672982 2026-02-13T19:03:33.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982' 2026-02-13T19:03:33.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:33.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542166 2026-02-13T19:03:33.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542166 2026-02-13T19:03:33.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166' 2026-02-13T19:03:33.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.557 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:33.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579914 2026-02-13T19:03:33.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579914 2026-02-13T19:03:33.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914' 2026-02-13T19:03:33.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:33.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247829 2026-02-13T19:03:33.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247829 2026-02-13T19:03:33.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829' 2026-02-13T19:03:33.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.686 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:33.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383687 2026-02-13T19:03:33.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383687 2026-02-13T19:03:33.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687' 2026-02-13T19:03:33.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.750 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:33.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958531 2026-02-13T19:03:33.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958531 2026-02-13T19:03:33.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687 6-399431958531' 2026-02-13T19:03:33.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.816 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:33.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757267 2026-02-13T19:03:33.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757267 2026-02-13T19:03:33.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687 6-399431958531 7-163208757267' 2026-02-13T19:03:33.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.881 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:33.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593748 2026-02-13T19:03:33.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593748 2026-02-13T19:03:33.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687 6-399431958531 7-163208757267 8-184683593748' 2026-02-13T19:03:33.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:33.945 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:34.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462930 2026-02-13T19:03:34.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462930 2026-02-13T19:03:34.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687 6-399431958531 7-163208757267 8-184683593748 9-201863462930' 2026-02-13T19:03:34.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:34.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:34.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299410 2026-02-13T19:03:34.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299410 2026-02-13T19:03:34.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-42949672982 2-60129542166 3-317827579914 4-98784247829 5-343597383687 6-399431958531 7-163208757267 8-184683593748 9-201863462930 10-223338299410' 2026-02-13T19:03:34.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:34.073 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836503 2026-02-13T19:03:34.073 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:34.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:34.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-13T19:03:34.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:34.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836503 2026-02-13T19:03:34.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-13T19:03:34.075 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 21474836503 2026-02-13T19:03:34.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:34.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836500 -lt 21474836503 2026-02-13T19:03:34.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:03:35.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:03:35.302 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:35.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-13T19:03:35.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:35.526 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672982 2026-02-13T19:03:35.526 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:35.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:35.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672982 2026-02-13T19:03:35.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:35.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672982 2026-02-13T19:03:35.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672982' 2026-02-13T19:03:35.528 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 42949672982 2026-02-13T19:03:35.529 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:35.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672982 -lt 42949672982 2026-02-13T19:03:35.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:35.754 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542166 2026-02-13T19:03:35.754 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:35.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:35.756 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542166 2026-02-13T19:03:35.756 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:35.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542166 2026-02-13T19:03:35.757 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542166' 2026-02-13T19:03:35.757 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 60129542166 2026-02-13T19:03:35.757 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:35.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542166 -lt 60129542166 2026-02-13T19:03:35.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:35.981 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579914 2026-02-13T19:03:35.981 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:35.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:35.983 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579914 2026-02-13T19:03:35.983 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:35.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579914 2026-02-13T19:03:35.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579914' 2026-02-13T19:03:35.984 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 317827579914 2026-02-13T19:03:35.984 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:36.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579915 -lt 317827579914 2026-02-13T19:03:36.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:36.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247829 2026-02-13T19:03:36.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:36.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:36.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247829 2026-02-13T19:03:36.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:36.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247829 2026-02-13T19:03:36.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247829' 2026-02-13T19:03:36.211 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.4 seq 98784247829 2026-02-13T19:03:36.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:36.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247829 -lt 98784247829 2026-02-13T19:03:36.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:36.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383687 2026-02-13T19:03:36.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:36.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:36.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383687 2026-02-13T19:03:36.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:36.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383687 2026-02-13T19:03:36.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383687' 2026-02-13T19:03:36.443 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.5 seq 343597383687 2026-02-13T19:03:36.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:36.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383687 -lt 343597383687 2026-02-13T19:03:36.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:36.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-399431958531 2026-02-13T19:03:36.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:36.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:36.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-399431958531 2026-02-13T19:03:36.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:36.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958531 2026-02-13T19:03:36.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 399431958531' 2026-02-13T19:03:36.669 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.6 seq 399431958531 2026-02-13T19:03:36.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:36.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958531 -lt 399431958531 2026-02-13T19:03:36.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:36.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757267 2026-02-13T19:03:36.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:36.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:36.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757267 2026-02-13T19:03:36.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:36.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757267 2026-02-13T19:03:36.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757267' 2026-02-13T19:03:36.895 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.7 seq 163208757267 2026-02-13T19:03:36.895 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:37.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757268 -lt 163208757267 2026-02-13T19:03:37.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:37.118 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593748 2026-02-13T19:03:37.118 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:37.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:37.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593748 2026-02-13T19:03:37.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:37.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593748 2026-02-13T19:03:37.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593748' 2026-02-13T19:03:37.120 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.8 seq 184683593748 2026-02-13T19:03:37.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:37.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593748 -lt 184683593748 2026-02-13T19:03:37.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:37.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-201863462930 2026-02-13T19:03:37.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:37.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:37.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-201863462930 2026-02-13T19:03:37.353 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:37.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462930 2026-02-13T19:03:37.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 201863462930' 2026-02-13T19:03:37.355 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.9 seq 201863462930 2026-02-13T19:03:37.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:37.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462931 -lt 201863462930 2026-02-13T19:03:37.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:37.583 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299410 2026-02-13T19:03:37.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:37.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:37.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299410 2026-02-13T19:03:37.585 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:37.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299410 2026-02-13T19:03:37.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299410' 2026-02-13T19:03:37.586 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.10 seq 223338299410 2026-02-13T19:03:37.586 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:37.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299410 -lt 223338299410 2026-02-13T19:03:37.812 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:37.812 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:37.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:38.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:38.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:38.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-13T19:03:38.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:38.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:38.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:38.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-13T19:03:38.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:38.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:38.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:03:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:03:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-13T19:03:38.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-13T19:03:39.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-13T19:03:39.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-13T19:03:39.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:03:39.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:start osd.9 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=2cb50084-c839-4a9d-a027-bd638a3bcfe2 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.66758/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:03:39.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-13T19:03:39.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-13T19:03:39.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:03:39.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:03:39.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:03:39.083 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:39.082+0000 7f1d5c047780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:39.084 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:39.083+0000 7f1d5c047780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:39.085 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:39.084+0000 7f1d5c047780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:03:39.209 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:39.208+0000 7f1d5c047780 -1 Falling back to public interface 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:39.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:03:39.298 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:39.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:39.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-13T19:03:39.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:03:39.542 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:03:39.541+0000 7f1d5c047780 -1 osd.9 94 log_to_monitors true 2026-02-13T19:03:40.532 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:03:40.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:03:40.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:03:40.533 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:40.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:03:40.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:osd.9 up in weight 1 up_from 98 up_thru 98 down_at 95 last_clean_interval [47,94) [v2:127.0.0.1:6874/398320777,v1:127.0.0.1:6875/398320777] [v2:127.0.0.1:6876/398320777,v1:127.0.0.1:6877/398320777] exists,up 778d9f16-96a3-415f-80ee-4531c69f5f3c 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:03:40.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:03:40.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:03:40.841 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:03:41.069 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.070 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:03:41.132 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836506 2026-02-13T19:03:41.132 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836506 2026-02-13T19:03:41.133 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506' 2026-02-13T19:03:41.133 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.133 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:03:41.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672986 2026-02-13T19:03:41.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672986 2026-02-13T19:03:41.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986' 2026-02-13T19:03:41.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:03:41.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542169 2026-02-13T19:03:41.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542169 2026-02-13T19:03:41.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169' 2026-02-13T19:03:41.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.264 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:03:41.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579918 2026-02-13T19:03:41.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579918 2026-02-13T19:03:41.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918' 2026-02-13T19:03:41.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:03:41.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247832 2026-02-13T19:03:41.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247832 2026-02-13T19:03:41.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832' 2026-02-13T19:03:41.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:03:41.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383690 2026-02-13T19:03:41.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383690 2026-02-13T19:03:41.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690' 2026-02-13T19:03:41.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.455 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:03:41.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958534 2026-02-13T19:03:41.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958534 2026-02-13T19:03:41.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690 6-399431958534' 2026-02-13T19:03:41.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.519 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:03:41.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757271 2026-02-13T19:03:41.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757271 2026-02-13T19:03:41.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690 6-399431958534 7-163208757271' 2026-02-13T19:03:41.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:03:41.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593751 2026-02-13T19:03:41.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593751 2026-02-13T19:03:41.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690 6-399431958534 7-163208757271 8-184683593751' 2026-02-13T19:03:41.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.650 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:03:41.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795011 2026-02-13T19:03:41.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795011 2026-02-13T19:03:41.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690 6-399431958534 7-163208757271 8-184683593751 9-420906795011' 2026-02-13T19:03:41.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:03:41.714 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299413 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299413 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-42949672986 2-60129542169 3-317827579918 4-98784247832 5-343597383690 6-399431958534 7-163208757271 8-184683593751 9-420906795011 10-223338299413' 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836506 2026-02-13T19:03:41.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:41.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:03:41.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836506 2026-02-13T19:03:41.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:41.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836506 2026-02-13T19:03:41.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2026-02-13T19:03:41.781 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 21474836506 2026-02-13T19:03:41.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:42.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836504 -lt 21474836506 2026-02-13T19:03:42.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:03:43.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:03:43.010 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:03:43.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836506 2026-02-13T19:03:43.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:43.236 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672986 2026-02-13T19:03:43.236 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:43.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:03:43.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672986 2026-02-13T19:03:43.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:43.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672986 2026-02-13T19:03:43.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672986' 2026-02-13T19:03:43.238 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 42949672986 2026-02-13T19:03:43.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:03:43.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672986 -lt 42949672986 2026-02-13T19:03:43.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:43.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542169 2026-02-13T19:03:43.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:43.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:03:43.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542169 2026-02-13T19:03:43.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:43.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542169 2026-02-13T19:03:43.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542169' 2026-02-13T19:03:43.467 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 60129542169 2026-02-13T19:03:43.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:03:43.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542169 -lt 60129542169 2026-02-13T19:03:43.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:43.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579918 2026-02-13T19:03:43.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:43.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:03:43.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579918 2026-02-13T19:03:43.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579918 2026-02-13T19:03:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579918' 2026-02-13T19:03:43.698 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 317827579918 2026-02-13T19:03:43.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:03:43.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579918 -lt 317827579918 2026-02-13T19:03:43.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:43.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247832 2026-02-13T19:03:43.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:03:43.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247832 2026-02-13T19:03:43.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:43.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247832 2026-02-13T19:03:43.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247832' 2026-02-13T19:03:43.931 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.4 seq 98784247832 2026-02-13T19:03:43.932 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:03:44.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247832 -lt 98784247832 2026-02-13T19:03:44.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383690 2026-02-13T19:03:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:44.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:03:44.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383690 2026-02-13T19:03:44.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:44.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383690 2026-02-13T19:03:44.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383690' 2026-02-13T19:03:44.160 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.5 seq 343597383690 2026-02-13T19:03:44.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:03:44.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383691 -lt 343597383690 2026-02-13T19:03:44.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:44.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-399431958534 2026-02-13T19:03:44.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:44.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:03:44.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-399431958534 2026-02-13T19:03:44.391 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:44.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958534 2026-02-13T19:03:44.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 399431958534' 2026-02-13T19:03:44.392 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.6 seq 399431958534 2026-02-13T19:03:44.392 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:03:44.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958535 -lt 399431958534 2026-02-13T19:03:44.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:44.624 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757271 2026-02-13T19:03:44.624 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:44.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:03:44.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757271 2026-02-13T19:03:44.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:44.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757271 2026-02-13T19:03:44.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757271' 2026-02-13T19:03:44.626 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.7 seq 163208757271 2026-02-13T19:03:44.627 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:03:44.856 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757271 -lt 163208757271 2026-02-13T19:03:44.856 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:44.857 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593751 2026-02-13T19:03:44.857 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:44.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:03:44.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593751 2026-02-13T19:03:44.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:44.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593751 2026-02-13T19:03:44.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593751' 2026-02-13T19:03:44.859 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.8 seq 184683593751 2026-02-13T19:03:44.860 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:03:45.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593752 -lt 184683593751 2026-02-13T19:03:45.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:45.091 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795011 2026-02-13T19:03:45.091 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:45.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:03:45.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795011 2026-02-13T19:03:45.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:45.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795011 2026-02-13T19:03:45.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795011' 2026-02-13T19:03:45.094 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.9 seq 420906795011 2026-02-13T19:03:45.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:03:45.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795012 -lt 420906795011 2026-02-13T19:03:45.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:03:45.319 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299413 2026-02-13T19:03:45.319 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:03:45.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:03:45.320 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299413 2026-02-13T19:03:45.321 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:03:45.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299413 2026-02-13T19:03:45.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299413' 2026-02-13T19:03:45.322 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.10 seq 223338299413 2026-02-13T19:03:45.322 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:03:45.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299414 -lt 223338299413 2026-02-13T19:03:45.549 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:03:45.549 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:45.549 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:45.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-13T19:03:45.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:03:45.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:03:45.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:03:45.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:03:45.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:03:45.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:03:45.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:03:46.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-13T19:03:46.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:03:46.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:03:46.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-13T19:03:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-13T19:03:46.697 INFO:tasks.workunit.client.0.trial034.stderr:pool 'remap-pool' does not exist 2026-02-13T19:03:46.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-13T19:03:46.987 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile remap-profile does not exist 2026-02-13T19:03:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:03:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-13T19:03:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-13T19:03:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-13T19:03:46.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-13T19:03:46.998 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-13T19:03:47.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-13T19:03:47.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-13T19:03:47.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-13T19:03:47.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-13T19:03:47.567 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-13T19:03:47.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-13T19:03:47.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-13T19:03:47.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-13T19:03:47.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-13T19:03:47.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-13T19:03:47.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-13T19:03:48.240 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-isa' already exists 2026-02-13T19:03:48.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:49.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:03:49.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:49.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:03:49.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:49.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:03:49.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:03:49.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:03:49.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:03:49.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:03:49.287 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-13T19:03:49.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-13T19:03:49.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:03:49.288 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-13T19:03:49.288 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:10 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-13T19:03:49.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-13T19:03:49.767 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already out. 2026-02-13T19:03:49.775 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:03:54.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-13T19:03:54.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-13T19:03:54.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:03:54.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-13T19:03:54.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-13T19:03:54.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:10 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:1' 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-13T19:03:55.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:03:55.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:03:55.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-13T19:03:55.256 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already in. 2026-02-13T19:03:55.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:03:55.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-13T19:03:55.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-13T19:03:55.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-13T19:03:55.554 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-isa' does not exist 2026-02-13T19:03:55.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:03:55.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-13T19:03:55.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-13T19:03:55.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-13T19:03:55.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:03:55.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:03:55.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:03:55.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:03:55.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:03:55.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-13T19:03:55.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-13T19:03:55.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:03:55.602 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-13T19:03:55.602 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:03:55.832 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:9' 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-13T19:03:55.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-13T19:03:56.080 INFO:tasks.workunit.client.0.trial034.stderr:osd.9 is already out. 2026-02-13T19:03:56.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:04:01.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-13T19:04:01.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-13T19:04:01.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:01.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-13T19:04:01.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-13T19:04:01.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:01.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-13T19:04:01.319 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:04:01.319 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:01.319 INFO:tasks.workunit.client.0.trial034.stderr:5' 2026-02-13T19:04:01.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-13T19:04:01.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:01.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:01.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-13T19:04:01.567 INFO:tasks.workunit.client.0.trial034.stderr:osd.9 is already in. 2026-02-13T19:04:01.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:01.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-13T19:04:01.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-13T19:04:01.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-13T19:04:01.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-13T19:04:01.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-13T19:04:02.257 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:04:02.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:04:03.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:04:03.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:04:03.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:03.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:03.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:04:03.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:03.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:03.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:03.310 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:03.310 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:03.537 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:04:03.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-13T19:04:03.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-13T19:04:03.786 INFO:tasks.workunit.client.0.trial034.stderr:osd.10 is already out. 2026-02-13T19:04:03.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:04:08.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:08.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:08.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:08.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-13T19:04:08.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:08.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:09.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:09.026 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:09.026 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:09.026 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:09.027 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:09.027 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:04:09.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-13T19:04:09.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:09.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:09.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-13T19:04:09.278 INFO:tasks.workunit.client.0.trial034.stderr:osd.10 is already in. 2026-02-13T19:04:09.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-13T19:04:09.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-13T19:04:09.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:04:09.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:09.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:09.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-13T19:04:09.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-13T19:04:09.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:04:09.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:04:09.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:04:09.487 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:04:09.488 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:04:09.488 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:04:09.488 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:04:09.488 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:04:09.488 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:04:09.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:04:09.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:04:09.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:04:09.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:04:09.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:04:09.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:09.793 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:09.794 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:04:09.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-13T19:04:09.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-13T19:04:09.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514' 2026-02-13T19:04:09.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:09.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:04:09.922 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672993 2026-02-13T19:04:09.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672993 2026-02-13T19:04:09.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993' 2026-02-13T19:04:09.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:09.923 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:04:09.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542177 2026-02-13T19:04:09.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542177 2026-02-13T19:04:09.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177' 2026-02-13T19:04:09.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:09.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:04:10.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579926 2026-02-13T19:04:10.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579926 2026-02-13T19:04:10.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926' 2026-02-13T19:04:10.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.052 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:04:10.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247840 2026-02-13T19:04:10.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247840 2026-02-13T19:04:10.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840' 2026-02-13T19:04:10.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:04:10.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383698 2026-02-13T19:04:10.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383698 2026-02-13T19:04:10.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698' 2026-02-13T19:04:10.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:04:10.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958542 2026-02-13T19:04:10.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958542 2026-02-13T19:04:10.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698 6-399431958542' 2026-02-13T19:04:10.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:04:10.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757278 2026-02-13T19:04:10.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757278 2026-02-13T19:04:10.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698 6-399431958542 7-163208757278' 2026-02-13T19:04:10.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:04:10.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593759 2026-02-13T19:04:10.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593759 2026-02-13T19:04:10.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698 6-399431958542 7-163208757278 8-184683593759' 2026-02-13T19:04:10.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:04:10.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795019 2026-02-13T19:04:10.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795019 2026-02-13T19:04:10.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698 6-399431958542 7-163208757278 8-184683593759 9-420906795019' 2026-02-13T19:04:10.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:10.440 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:04:10.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299421 2026-02-13T19:04:10.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299421 2026-02-13T19:04:10.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-42949672993 2-60129542177 3-317827579926 4-98784247840 5-343597383698 6-399431958542 7-163208757278 8-184683593759 9-420906795019 10-223338299421' 2026-02-13T19:04:10.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:10.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836514 2026-02-13T19:04:10.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:10.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:04:10.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-13T19:04:10.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:10.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836514 2026-02-13T19:04:10.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-13T19:04:10.506 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836514 2026-02-13T19:04:10.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:04:10.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-13T19:04:10.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:10.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672993 2026-02-13T19:04:10.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:10.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:04:10.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672993 2026-02-13T19:04:10.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:10.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672993 2026-02-13T19:04:10.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672993' 2026-02-13T19:04:10.740 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949672993 2026-02-13T19:04:10.740 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:04:10.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672994 -lt 42949672993 2026-02-13T19:04:10.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:10.965 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542177 2026-02-13T19:04:10.965 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:10.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:04:10.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542177 2026-02-13T19:04:10.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:10.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542177 2026-02-13T19:04:10.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542177' 2026-02-13T19:04:10.969 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542177 2026-02-13T19:04:10.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:04:11.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542177 -lt 60129542177 2026-02-13T19:04:11.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:11.196 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579926 2026-02-13T19:04:11.196 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:11.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:04:11.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579926 2026-02-13T19:04:11.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:11.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579926 2026-02-13T19:04:11.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579926' 2026-02-13T19:04:11.199 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 317827579926 2026-02-13T19:04:11.199 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:04:11.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579926 -lt 317827579926 2026-02-13T19:04:11.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:11.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247840 2026-02-13T19:04:11.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:11.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:04:11.423 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247840 2026-02-13T19:04:11.423 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:11.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247840 2026-02-13T19:04:11.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247840' 2026-02-13T19:04:11.425 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 98784247840 2026-02-13T19:04:11.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:04:11.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247840 -lt 98784247840 2026-02-13T19:04:11.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:11.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383698 2026-02-13T19:04:11.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:11.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:04:11.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383698 2026-02-13T19:04:11.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:11.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383698 2026-02-13T19:04:11.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383698' 2026-02-13T19:04:11.656 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 343597383698 2026-02-13T19:04:11.657 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:04:11.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383698 -lt 343597383698 2026-02-13T19:04:11.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:11.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-399431958542 2026-02-13T19:04:11.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:11.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:04:11.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-399431958542 2026-02-13T19:04:11.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:11.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958542 2026-02-13T19:04:11.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 399431958542' 2026-02-13T19:04:11.892 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 399431958542 2026-02-13T19:04:11.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:04:12.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958542 -lt 399431958542 2026-02-13T19:04:12.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:12.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757278 2026-02-13T19:04:12.117 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:12.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:04:12.118 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757278 2026-02-13T19:04:12.118 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:12.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757278 2026-02-13T19:04:12.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757278' 2026-02-13T19:04:12.120 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 163208757278 2026-02-13T19:04:12.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:04:12.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757278 -lt 163208757278 2026-02-13T19:04:12.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:12.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593759 2026-02-13T19:04:12.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:12.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:04:12.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593759 2026-02-13T19:04:12.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:12.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593759 2026-02-13T19:04:12.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593759' 2026-02-13T19:04:12.347 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 184683593759 2026-02-13T19:04:12.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:04:12.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593759 -lt 184683593759 2026-02-13T19:04:12.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:12.570 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795019 2026-02-13T19:04:12.570 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:12.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:04:12.571 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795019 2026-02-13T19:04:12.571 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:12.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795019 2026-02-13T19:04:12.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795019' 2026-02-13T19:04:12.573 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.9 seq 420906795019 2026-02-13T19:04:12.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:04:12.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795019 -lt 420906795019 2026-02-13T19:04:12.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:12.796 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299421 2026-02-13T19:04:12.796 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:12.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:04:12.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299421 2026-02-13T19:04:12.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:12.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299421 2026-02-13T19:04:12.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299421' 2026-02-13T19:04:12.800 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.10 seq 223338299421 2026-02-13T19:04:12.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:04:13.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299421 -lt 223338299421 2026-02-13T19:04:13.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:04:13.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:13.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:13.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-13T19:04:13.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:04:13.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:04:13.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-13T19:04:13.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:04:13.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:13.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:13.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-13T19:04:13.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:04:13.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:04:13.878 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:13.879 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:13.879 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:13.879 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:13.879 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:14.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:14.105 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:14.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-13T19:04:14.355 INFO:tasks.workunit.client.0.trial034.stderr:osd.5 is already out. 2026-02-13T19:04:14.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:14.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-13T19:04:14.661 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 is already out. 2026-02-13T19:04:14.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-13T19:04:14.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:04:14.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:04:14.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:04:14.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:04:14.753 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:04:14.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:04:14.977 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:04:14.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:14.979 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:04:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-13T19:04:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-13T19:04:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517' 2026-02-13T19:04:15.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:04:15.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672997 2026-02-13T19:04:15.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672997 2026-02-13T19:04:15.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997' 2026-02-13T19:04:15.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:04:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542180 2026-02-13T19:04:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542180 2026-02-13T19:04:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180' 2026-02-13T19:04:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.172 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:04:15.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579929 2026-02-13T19:04:15.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579929 2026-02-13T19:04:15.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929' 2026-02-13T19:04:15.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.235 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:04:15.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247843 2026-02-13T19:04:15.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247843 2026-02-13T19:04:15.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843' 2026-02-13T19:04:15.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:04:15.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383701 2026-02-13T19:04:15.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383701 2026-02-13T19:04:15.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701' 2026-02-13T19:04:15.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:04:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958545 2026-02-13T19:04:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958545 2026-02-13T19:04:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701 6-399431958545' 2026-02-13T19:04:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:04:15.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757281 2026-02-13T19:04:15.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757281 2026-02-13T19:04:15.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701 6-399431958545 7-163208757281' 2026-02-13T19:04:15.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:04:15.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593762 2026-02-13T19:04:15.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593762 2026-02-13T19:04:15.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701 6-399431958545 7-163208757281 8-184683593762' 2026-02-13T19:04:15.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.557 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:04:15.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795022 2026-02-13T19:04:15.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795022 2026-02-13T19:04:15.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701 6-399431958545 7-163208757281 8-184683593762 9-420906795022' 2026-02-13T19:04:15.622 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:15.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:04:15.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299424 2026-02-13T19:04:15.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299424 2026-02-13T19:04:15.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-42949672997 2-60129542180 3-317827579929 4-98784247843 5-343597383701 6-399431958545 7-163208757281 8-184683593762 9-420906795022 10-223338299424' 2026-02-13T19:04:15.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:15.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836517 2026-02-13T19:04:15.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:15.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:04:15.689 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-13T19:04:15.689 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:15.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836517 2026-02-13T19:04:15.690 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836517 2026-02-13T19:04:15.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-13T19:04:15.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:04:15.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-13T19:04:15.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:04:16.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:04:16.917 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:04:17.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-13T19:04:17.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:17.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672997 2026-02-13T19:04:17.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:17.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:04:17.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672997 2026-02-13T19:04:17.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:17.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672997 2026-02-13T19:04:17.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672997' 2026-02-13T19:04:17.152 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949672997 2026-02-13T19:04:17.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:04:17.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672997 -lt 42949672997 2026-02-13T19:04:17.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:17.379 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542180 2026-02-13T19:04:17.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:17.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:04:17.381 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542180 2026-02-13T19:04:17.381 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:17.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542180 2026-02-13T19:04:17.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-13T19:04:17.382 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542180 2026-02-13T19:04:17.383 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:04:17.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542180 -lt 60129542180 2026-02-13T19:04:17.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:17.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579929 2026-02-13T19:04:17.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:17.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:04:17.620 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579929 2026-02-13T19:04:17.620 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:17.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579929 2026-02-13T19:04:17.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579929' 2026-02-13T19:04:17.621 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 317827579929 2026-02-13T19:04:17.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:04:17.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579929 -lt 317827579929 2026-02-13T19:04:17.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:17.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247843 2026-02-13T19:04:17.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:17.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:04:17.854 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247843 2026-02-13T19:04:17.854 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:17.855 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247843 2026-02-13T19:04:17.855 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247843' 2026-02-13T19:04:17.855 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 98784247843 2026-02-13T19:04:17.856 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:04:18.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247843 -lt 98784247843 2026-02-13T19:04:18.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:18.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383701 2026-02-13T19:04:18.088 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:18.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:04:18.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383701 2026-02-13T19:04:18.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:18.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383701 2026-02-13T19:04:18.091 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 343597383701 2026-02-13T19:04:18.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383701' 2026-02-13T19:04:18.091 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:04:18.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383701 -lt 343597383701 2026-02-13T19:04:18.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:18.318 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-399431958545 2026-02-13T19:04:18.318 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:18.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:04:18.319 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-399431958545 2026-02-13T19:04:18.320 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:18.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958545 2026-02-13T19:04:18.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 399431958545' 2026-02-13T19:04:18.321 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 399431958545 2026-02-13T19:04:18.321 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:04:18.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958545 -lt 399431958545 2026-02-13T19:04:18.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:18.549 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757281 2026-02-13T19:04:18.549 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:18.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:04:18.551 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757281 2026-02-13T19:04:18.551 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:18.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757281 2026-02-13T19:04:18.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757281' 2026-02-13T19:04:18.553 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 163208757281 2026-02-13T19:04:18.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:04:18.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757282 -lt 163208757281 2026-02-13T19:04:18.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:18.777 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593762 2026-02-13T19:04:18.777 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:18.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:04:18.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593762 2026-02-13T19:04:18.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:18.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593762 2026-02-13T19:04:18.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593762' 2026-02-13T19:04:18.781 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 184683593762 2026-02-13T19:04:18.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:04:19.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593763 -lt 184683593762 2026-02-13T19:04:19.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:19.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795022 2026-02-13T19:04:19.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:19.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:04:19.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795022 2026-02-13T19:04:19.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:19.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795022 2026-02-13T19:04:19.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795022' 2026-02-13T19:04:19.009 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.9 seq 420906795022 2026-02-13T19:04:19.010 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:04:19.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795022 -lt 420906795022 2026-02-13T19:04:19.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:19.235 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299424 2026-02-13T19:04:19.235 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:19.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:04:19.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299424 2026-02-13T19:04:19.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:19.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299424 2026-02-13T19:04:19.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299424' 2026-02-13T19:04:19.238 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.10 seq 223338299424 2026-02-13T19:04:19.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:04:19.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299425 -lt 223338299424 2026-02-13T19:04:19.466 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:04:19.466 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:19.466 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:19.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-13T19:04:19.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:04:19.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:04:19.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:04:19.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:04:19.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:04:19.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:04:19.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:04:20.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-13T19:04:20.012 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:04:20.012 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:20.012 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:20.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-13T19:04:20.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:04:20.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:20.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:20.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-13T19:04:20.570 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:20.570 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:20.571 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:20.571 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:20.571 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:20.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-13T19:04:20.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:20.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:20.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:20.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:20.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-13T19:04:20.572 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:20.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:20.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-13T19:04:20.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:20.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:20.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:20.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-13T19:04:21.095 INFO:tasks.workunit.client.0.trial034.stderr:osd.5 is already in. 2026-02-13T19:04:21.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-13T19:04:21.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-13T19:04:21.382 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 is already in. 2026-02-13T19:04:21.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-13T19:04:21.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:04:21.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:04:21.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:04:21.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:04:21.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:04:21.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:04:21.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:04:21.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:04:21.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:04:21.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:04:21.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:8 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:21.702 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:21.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:04:21.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:21.703 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:04:21.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836520 2026-02-13T19:04:21.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836520 2026-02-13T19:04:21.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520' 2026-02-13T19:04:21.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:21.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:04:21.832 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949673000 2026-02-13T19:04:21.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949673000 2026-02-13T19:04:21.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000' 2026-02-13T19:04:21.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:21.833 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:04:21.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542183 2026-02-13T19:04:21.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542183 2026-02-13T19:04:21.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183' 2026-02-13T19:04:21.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:21.897 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:04:21.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579932 2026-02-13T19:04:21.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579932 2026-02-13T19:04:21.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932' 2026-02-13T19:04:21.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:21.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:04:22.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=98784247846 2026-02-13T19:04:22.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 98784247846 2026-02-13T19:04:22.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846' 2026-02-13T19:04:22.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.025 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:04:22.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383705 2026-02-13T19:04:22.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383705 2026-02-13T19:04:22.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705' 2026-02-13T19:04:22.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:04:22.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=399431958549 2026-02-13T19:04:22.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 399431958549 2026-02-13T19:04:22.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705 6-399431958549' 2026-02-13T19:04:22.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:04:22.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757285 2026-02-13T19:04:22.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757285 2026-02-13T19:04:22.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705 6-399431958549 7-163208757285' 2026-02-13T19:04:22.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.221 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:04:22.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593765 2026-02-13T19:04:22.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593765 2026-02-13T19:04:22.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705 6-399431958549 7-163208757285 8-184683593765' 2026-02-13T19:04:22.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.283 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-13T19:04:22.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795025 2026-02-13T19:04:22.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795025 2026-02-13T19:04:22.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705 6-399431958549 7-163208757285 8-184683593765 9-420906795025' 2026-02-13T19:04:22.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:04:22.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-13T19:04:22.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299427 2026-02-13T19:04:22.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299427 2026-02-13T19:04:22.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836520 1-42949673000 2-60129542183 3-317827579932 4-98784247846 5-343597383705 6-399431958549 7-163208757285 8-184683593765 9-420906795025 10-223338299427' 2026-02-13T19:04:22.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:22.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836520 2026-02-13T19:04:22.411 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:22.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:04:22.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836520 2026-02-13T19:04:22.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:22.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836520 2026-02-13T19:04:22.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-13T19:04:22.413 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 21474836520 2026-02-13T19:04:22.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:04:22.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836518 -lt 21474836520 2026-02-13T19:04:22.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:04:23.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:04:23.636 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:04:23.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-13T19:04:23.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:23.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949673000 2026-02-13T19:04:23.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:23.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:04:23.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949673000 2026-02-13T19:04:23.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:23.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949673000 2026-02-13T19:04:23.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949673000' 2026-02-13T19:04:23.872 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 42949673000 2026-02-13T19:04:23.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:04:24.101 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949673000 -lt 42949673000 2026-02-13T19:04:24.101 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:24.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542183 2026-02-13T19:04:24.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:24.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:04:24.104 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542183 2026-02-13T19:04:24.104 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:24.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542183 2026-02-13T19:04:24.105 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 60129542183 2026-02-13T19:04:24.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542183' 2026-02-13T19:04:24.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:04:24.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542183 -lt 60129542183 2026-02-13T19:04:24.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:24.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579932 2026-02-13T19:04:24.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:24.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:04:24.331 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579932 2026-02-13T19:04:24.331 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:24.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579932 2026-02-13T19:04:24.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579932' 2026-02-13T19:04:24.332 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 317827579932 2026-02-13T19:04:24.333 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:04:24.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579932 -lt 317827579932 2026-02-13T19:04:24.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:24.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-98784247846 2026-02-13T19:04:24.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:24.561 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:04:24.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-98784247846 2026-02-13T19:04:24.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:24.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=98784247846 2026-02-13T19:04:24.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 98784247846' 2026-02-13T19:04:24.564 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 98784247846 2026-02-13T19:04:24.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:04:24.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 98784247847 -lt 98784247846 2026-02-13T19:04:24.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:24.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-343597383705 2026-02-13T19:04:24.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:24.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:04:24.790 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-343597383705 2026-02-13T19:04:24.790 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:24.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383705 2026-02-13T19:04:24.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 343597383705' 2026-02-13T19:04:24.792 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 343597383705 2026-02-13T19:04:24.792 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:04:25.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383705 -lt 343597383705 2026-02-13T19:04:25.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:25.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-399431958549 2026-02-13T19:04:25.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:25.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:04:25.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-399431958549 2026-02-13T19:04:25.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:25.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=399431958549 2026-02-13T19:04:25.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 399431958549' 2026-02-13T19:04:25.022 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 399431958549 2026-02-13T19:04:25.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:04:25.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 399431958549 -lt 399431958549 2026-02-13T19:04:25.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:25.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-163208757285 2026-02-13T19:04:25.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:25.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:04:25.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757285 2026-02-13T19:04:25.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:25.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757285 2026-02-13T19:04:25.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 163208757285' 2026-02-13T19:04:25.248 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 163208757285 2026-02-13T19:04:25.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:04:25.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757285 -lt 163208757285 2026-02-13T19:04:25.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:25.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-184683593765 2026-02-13T19:04:25.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:25.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:04:25.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-184683593765 2026-02-13T19:04:25.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:25.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593765 2026-02-13T19:04:25.480 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 184683593765 2026-02-13T19:04:25.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 184683593765' 2026-02-13T19:04:25.481 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:04:25.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593766 -lt 184683593765 2026-02-13T19:04:25.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:25.712 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795025 2026-02-13T19:04:25.712 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:25.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-13T19:04:25.713 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795025 2026-02-13T19:04:25.713 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:25.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795025 2026-02-13T19:04:25.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795025' 2026-02-13T19:04:25.715 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.9 seq 420906795025 2026-02-13T19:04:25.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-13T19:04:25.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795026 -lt 420906795025 2026-02-13T19:04:25.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:04:25.945 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-223338299427 2026-02-13T19:04:25.946 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:04:25.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-13T19:04:25.947 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-223338299427 2026-02-13T19:04:25.947 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:04:25.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299427 2026-02-13T19:04:25.948 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.10 seq 223338299427 2026-02-13T19:04:25.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 223338299427' 2026-02-13T19:04:25.949 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-13T19:04:26.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299428 -lt 223338299427 2026-02-13T19:04:26.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:04:26.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:26.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:26.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-13T19:04:26.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:04:26.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:04:26.719 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-13T19:04:26.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:04:26.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:04:26.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:04:27.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-13T19:04:27.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:04:27.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:04:27.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-13T19:04:27.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:04:27.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:27.037 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-13T19:04:27.037 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:10' 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-13T19:04:27.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-13T19:04:27.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:27.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-13T19:04:27.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-13T19:04:27.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:04:27.513 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:04:27.522 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-13T19:04:27.815 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-13T19:04:27.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-13T19:04:28.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-13T19:04:28.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-13T19:04:28.498 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-lrc-a' already exists 2026-02-13T19:04:28.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:04:29.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:04:29.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:29.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:29.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:04:29.548 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-13T19:04:29.548 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-13T19:04:29.548 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:29.548 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:29.548 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-13T19:04:29.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-13T19:04:29.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-13T19:04:30.031 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 is already out. 2026-02-13T19:04:30.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:04:35.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-13T19:04:35.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-13T19:04:35.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:35.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-13T19:04:35.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-13T19:04:35.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:35.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-13T19:04:35.270 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:35.270 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:04:35.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-13T19:04:35.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:35.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:35.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-13T19:04:35.520 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 is already in. 2026-02-13T19:04:35.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:35.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-13T19:04:35.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-13T19:04:35.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-13T19:04:35.823 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-lrc-a' does not exist 2026-02-13T19:04:35.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-13T19:04:36.108 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-13T19:04:36.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-13T19:04:36.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-13T19:04:36.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-13T19:04:36.785 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-lrc' already exists 2026-02-13T19:04:36.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:37.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:04:37.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:37.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:04:37.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:04:37.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:37.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:37.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:04:37.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-13T19:04:37.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-13T19:04:37.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:37.836 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-13T19:04:37.836 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr:10 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:04:38.062 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-13T19:04:38.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-13T19:04:38.313 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already out. 2026-02-13T19:04:38.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:04:43.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-13T19:04:43.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-13T19:04:43.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-13T19:04:43.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:43.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-13T19:04:43.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:43.554 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:10 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:4' 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-13T19:04:43.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:43.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:43.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-13T19:04:43.804 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already in. 2026-02-13T19:04:43.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:43.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-13T19:04:43.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-13T19:04:43.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-13T19:04:44.105 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-lrc' does not exist 2026-02-13T19:04:44.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-13T19:04:44.396 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile profile-lrc does not exist 2026-02-13T19:04:44.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-13T19:04:44.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-13T19:04:44.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-13T19:04:44.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-13T19:04:44.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-13T19:04:44.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-13T19:04:44.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-13T19:04:44.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-13T19:04:45.083 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-shec' already exists 2026-02-13T19:04:45.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:04:46.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-13T19:04:46.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:46.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-13T19:04:46.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-13T19:04:46.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-13T19:04:46.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-13T19:04:46.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:46.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:46.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-13T19:04:46.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-13T19:04:46.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-13T19:04:46.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:46.423 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-13T19:04:46.423 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:46.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-13T19:04:46.648 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:46.648 INFO:tasks.workunit.client.0.trial034.stderr:1' 2026-02-13T19:04:46.649 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-13T19:04:46.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-13T19:04:46.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-13T19:04:46.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-13T19:04:46.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-13T19:04:46.926 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 is already out. 2026-02-13T19:04:46.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-13T19:04:51.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:04:52.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-13T19:04:52.160 INFO:tasks.workunit.client.0.trial034.stderr:9 2026-02-13T19:04:52.160 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:04:52.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-13T19:04:52.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-13T19:04:52.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-13T19:04:52.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-13T19:04:52.412 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 is already in. 2026-02-13T19:04:52.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-13T19:04:52.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-13T19:04:52.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-13T19:04:52.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-13T19:04:52.712 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-shec' does not exist 2026-02-13T19:04:52.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-13T19:04:52.998 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile profile-shec does not exist 2026-02-13T19:04:53.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-13T19:04:53.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-13T19:04:53.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-13T19:04:53.277 INFO:tasks.workunit.client.0.trial034.stderr:pool 'ecpool' does not exist 2026-02-13T19:04:53.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-13T19:04:53.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-13T19:04:53.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:04:53.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-13T19:04:53.286 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:04:53.286 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:04:53.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:04:53.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:04:53.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:04:53.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:04:53.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:04:53.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:04:53.401 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:04:53.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:04:53.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:04:53.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:04:53.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:04:53.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:04:53.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:04:53.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:04:53.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-13T19:04:53.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:04:53.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:04:53.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.66758 2026-02-13T19:04:53.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:04:53.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:04:53.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:04:53.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code 0 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:04:53.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:04:53.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:04:53.468 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:04:53.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:04:53.469 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:04:53.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:04:53.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:04:53.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:04:53.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:04:53.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:04:53.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:04:53.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:04:53.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-13T19:04:53.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:04:53.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.66758 2026-02-13T19:04:53.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.66758 2026-02-13T19:04:53.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:04:53.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:04:53.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:04:53.477 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:04:53.477 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:04:53.545 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-13T19:04:53.545 DEBUG:teuthology.orchestra.run.trial034:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-13T19:04:53.607 INFO:tasks.workunit.client.0.trial034.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-13T19:04:53.612 INFO:tasks.workunit.client.0.trial034.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-13T19:04:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:04:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-13T19:04:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-13T19:04:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-eio 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-13T19:04:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none ' 2026-02-13T19:04:53.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-13T19:04:53.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-13T19:04:53.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-13T19:04:53.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_object_attr_read_error 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_recovery_multiple_errors 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_recovery_multiple_objects 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_recovery_unfound 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_ec_single_recovery_error 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_bad_size_shard_0 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_bad_size_shard_1 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-13T19:04:53.616 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_subread_missing 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:04:53.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:04:53.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:04:53.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:04:53.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:04:53.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:04:53.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:04:53.620 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:04:53.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:04:53.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:04:53.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:04:53.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:04:53.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.623 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:04:53.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:04:53.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:04:53.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:04:53.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:04:53.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.625 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:04:53.626 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:04:53.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:04:53.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:04:53.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:04:53.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:04:53.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:04:53.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:53.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:04:53.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:04:53.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:04:53.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:04:53.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:04:53.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:04:53.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:04:53.672 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:04:53.672 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:04:53.672 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:04:53.672 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:04:53.673 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.673 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.673 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:04:53.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:04:53.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:04:53.715 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:04:53.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:04:53.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:04:53.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:04:53.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:04:53.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:04:53.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:04:53.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:04:53.861 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:04:53.862 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:53.862 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:53.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:53.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:04:53.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:04:53.877 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:04:53.877 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:04:53.975 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:04:53.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:04:54.985 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:04:54.985 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:04:54.985 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:04:54.985 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:04:54.985 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:54.986 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:54.986 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:04:54.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:04:54.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:04:55.023 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:04:55.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:04:55.028 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:04:53.662+0000 7f03efa33c00 0 load: jerasure load: lrc 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-13T19:04:55.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:04:55.031 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-13T19:04:55.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:04:55.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68bca720-b67e-42ac-9d49-008022945ef3 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 68bca720-b67e-42ac-9d49-008022945ef3' 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 68bca720-b67e-42ac-9d49-008022945ef3 2026-02-13T19:04:55.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:04:55.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDXdY9pxs6yAhAAxSuyi/yjKoDqKDPTMJhV4w== 2026-02-13T19:04:55.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDXdY9pxs6yAhAAxSuyi/yjKoDqKDPTMJhV4w=="}' 2026-02-13T19:04:55.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68bca720-b67e-42ac-9d49-008022945ef3 -i td/test-erasure-eio/0/new.json 2026-02-13T19:04:55.151 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:04:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:04:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDXdY9pxs6yAhAAxSuyi/yjKoDqKDPTMJhV4w== --osd-uuid 68bca720-b67e-42ac-9d49-008022945ef3 2026-02-13T19:04:55.177 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.176+0000 7f9d09436780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.178 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.178+0000 7f9d09436780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.179 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.179+0000 7f9d09436780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.180 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.179+0000 7f9d09436780 -1 bdev(0x2e86c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:04:55.180 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.179+0000 7f9d09436780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:04:55.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:04:55.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:04:55.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:04:55.592 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:04:55.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:04:55.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:04:55.697 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:04:55.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:04:55.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:04:55.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:04:55.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:04:55.714 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.713+0000 7f0e89e47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.716 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.715+0000 7f0e89e47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.717 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.716+0000 7f0e89e47780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:04:55.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:04:55.798 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:04:55.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:04:55.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:04:55.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:04:55.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:04:55.917 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:55.917+0000 7f0e89e47780 -1 Falling back to public interface 2026-02-13T19:04:56.174 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:56.173+0000 7f0e89e47780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:04:56.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:04:56.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:04:56.899 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:04:56.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:04:56.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:04:56.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:04:57.122 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3945974205,v1:127.0.0.1:6803/3945974205] [v2:127.0.0.1:6804/3945974205,v1:127.0.0.1:6805/3945974205] exists,up 68bca720-b67e-42ac-9d49-008022945ef3 2026-02-13T19:04:57.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:04:57.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:04:57.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:04:57.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:04:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:04:57.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=28a7894c-38da-436d-bdb4-565d12e70568 2026-02-13T19:04:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 28a7894c-38da-436d-bdb4-565d12e70568' 2026-02-13T19:04:57.130 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 28a7894c-38da-436d-bdb4-565d12e70568 2026-02-13T19:04:57.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:04:57.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDZdY9pZBIfCBAA3KZaCVxDrN7Qtp1QdrIM6w== 2026-02-13T19:04:57.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDZdY9pZBIfCBAA3KZaCVxDrN7Qtp1QdrIM6w=="}' 2026-02-13T19:04:57.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 28a7894c-38da-436d-bdb4-565d12e70568 -i td/test-erasure-eio/1/new.json 2026-02-13T19:04:57.371 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:04:57.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:04:57.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZdY9pZBIfCBAA3KZaCVxDrN7Qtp1QdrIM6w== --osd-uuid 28a7894c-38da-436d-bdb4-565d12e70568 2026-02-13T19:04:57.401 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:57.400+0000 7f6d62440780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:57.402 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:57.401+0000 7f6d62440780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:57.403 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:57.402+0000 7f6d62440780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:57.403 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:57.402+0000 7f6d62440780 -1 bdev(0x423ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:04:57.404 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:57.403+0000 7f6d62440780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:04:57.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:04:57.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:04:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:04:57.735 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:04:57.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:04:58.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:04:58.056 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:04:58.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:04:58.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:04:58.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:04:58.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:04:58.072 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:58.071+0000 7f51d8273780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:58.073 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:58.073+0000 7f51d8273780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:58.074 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:58.073+0000 7f51d8273780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:04:58.223 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:58.221+0000 7f51d8273780 -1 Falling back to public interface 2026-02-13T19:04:58.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:04:58.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:04:58.281 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:04:58.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:04:58.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:04:58.513 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:04:58.512+0000 7f51d8273780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:04:58.524 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:04:59.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:04:59.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:04:59.526 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:04:59.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:04:59.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:04:59.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:04:59.753 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2715004579,v1:127.0.0.1:6811/2715004579] [v2:127.0.0.1:6812/2715004579,v1:127.0.0.1:6813/2715004579] exists,up 28a7894c-38da-436d-bdb4-565d12e70568 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:04:59.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:04:59.759 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:04:59.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:04:59.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:04:59.762 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:04:59.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 509c9de8-03b1-40a7-8886-f5644db5d972' 2026-02-13T19:04:59.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:04:59.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDbdY9plcm5LRAAKnhN6blYjs0Yf43DMWPTTA== 2026-02-13T19:04:59.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDbdY9plcm5LRAAKnhN6blYjs0Yf43DMWPTTA=="}' 2026-02-13T19:04:59.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 509c9de8-03b1-40a7-8886-f5644db5d972 -i td/test-erasure-eio/2/new.json 2026-02-13T19:04:59.997 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:05:00.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:05:00.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDbdY9plcm5LRAAKnhN6blYjs0Yf43DMWPTTA== --osd-uuid 509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:05:00.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.023+0000 7ff041634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.025 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.024+0000 7ff041634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.026 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.025+0000 7ff041634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.026 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.025+0000 7ff041634780 -1 bdev(0x4196c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:05:00.026 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.025+0000 7ff041634780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:05:00.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:05:00.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:05:00.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:05:00.349 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:05:00.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:05:00.665 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:05:00.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:05:00.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:05:00.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:05:00.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:05:00.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:05:00.682 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.681+0000 7f6e16a45780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.684 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.683+0000 7f6e16a45780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.685 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.684+0000 7f6e16a45780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:00.812 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:00.811+0000 7f6e16a45780 -1 Falling back to public interface 2026-02-13T19:05:00.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:05:00.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:05:00.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:05:00.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:05:00.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:05:00.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:00.905 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:05:00.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:05:00.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:00.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:01.133 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:05:01.155 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:01.154+0000 7f6e16a45780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:05:02.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:05:02.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:02.135 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:05:02.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:05:02.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:02.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:02.365 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3192930055,v1:127.0.0.1:6819/3192930055] [v2:127.0.0.1:6820/3192930055,v1:127.0.0.1:6821/3192930055] exists,up 509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:05:02.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:05:02.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:05:02.371 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:05:02.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03' 2026-02-13T19:05:02.373 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03 2026-02-13T19:05:02.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:05:02.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDedY9p2xisFhAAXc1w+RwSkYfCjvfBg/Mqsg== 2026-02-13T19:05:02.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDedY9p2xisFhAAXc1w+RwSkYfCjvfBg/Mqsg=="}' 2026-02-13T19:05:02.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03 -i td/test-erasure-eio/3/new.json 2026-02-13T19:05:02.610 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:05:02.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:05:02.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDedY9p2xisFhAAXc1w+RwSkYfCjvfBg/Mqsg== --osd-uuid 4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03 2026-02-13T19:05:02.636 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:02.636+0000 7ff80485f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:02.638 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:02.637+0000 7ff80485f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:02.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:02.638+0000 7ff80485f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:02.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:02.638+0000 7ff80485f780 -1 bdev(0x2f5cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:05:02.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:02.638+0000 7ff80485f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:05:02.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:05:02.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:05:02.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:05:02.961 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:05:02.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:05:03.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:05:03.282 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:05:03.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:05:03.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:05:03.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:05:03.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:05:03.299 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:03.298+0000 7f934b02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:03.300 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:03.299+0000 7f934b02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:03.301 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:03.300+0000 7f934b02d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:03.438 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:03.436+0000 7f934b02d780 -1 Falling back to public interface 2026-02-13T19:05:03.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:05:03.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:05:03.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:05:03.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:05:03.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:05:03.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:03.509 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:05:03.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:05:03.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:03.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:05:03.709 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:03.708+0000 7f934b02d780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:05:03.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:05:04.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:05:04.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:04.737 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:05:04.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:05:04.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:04.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:05:04.965 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/667399724,v1:127.0.0.1:6827/667399724] [v2:127.0.0.1:6828/667399724,v1:127.0.0.1:6829/667399724] exists,up 4f770fb0-2cf4-4302-a01e-5ed9aa4c8e03 2026-02-13T19:05:04.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:05:04.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:05:04.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:05:04.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:05:04.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:05:04.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:05:04.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:05:04.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eeab3efe-17c0-483f-aa18-2341e971b58b 2026-02-13T19:05:04.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 eeab3efe-17c0-483f-aa18-2341e971b58b' 2026-02-13T19:05:04.972 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 eeab3efe-17c0-483f-aa18-2341e971b58b 2026-02-13T19:05:04.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:05:04.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgdY9pFQxeOhAAbpt1vf9ylZxKY3ToiOrJCA== 2026-02-13T19:05:04.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgdY9pFQxeOhAAbpt1vf9ylZxKY3ToiOrJCA=="}' 2026-02-13T19:05:04.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eeab3efe-17c0-483f-aa18-2341e971b58b -i td/test-erasure-eio/4/new.json 2026-02-13T19:05:05.222 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:05:05.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:05:05.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgdY9pFQxeOhAAbpt1vf9ylZxKY3ToiOrJCA== --osd-uuid eeab3efe-17c0-483f-aa18-2341e971b58b 2026-02-13T19:05:05.247 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.246+0000 7f9122041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:05.249 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.248+0000 7f9122041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:05.249 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.249+0000 7f9122041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:05.250 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.249+0000 7f9122041780 -1 bdev(0x3f70c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:05:05.250 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.249+0000 7f9122041780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:05:05.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:05:05.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:05:05.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:05:05.600 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:05:05.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:05:05.920 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:05:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:05:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:05:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:05:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:05:05.922 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:05:05.936 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.935+0000 7fccaf433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:05.937 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.937+0000 7fccaf433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:05.939 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:05.937+0000 7fccaf433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:06.075 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:06.074+0000 7fccaf433780 -1 Falling back to public interface 2026-02-13T19:05:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:05:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:05:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:05:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:05:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:05:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:06.151 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:05:06.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:05:06.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:06.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:05:06.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:05:06.379 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:06.377+0000 7fccaf433780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:05:07.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:05:07.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:07.377 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:05:07.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:05:07.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:07.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:05:07.601 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3899876164,v1:127.0.0.1:6835/3899876164] [v2:127.0.0.1:6836/3899876164,v1:127.0.0.1:6837/3899876164] exists,up eeab3efe-17c0-483f-aa18-2341e971b58b 2026-02-13T19:05:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:05:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:05:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:05:07.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:05:07.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:05:07.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:05:07.648 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:05:07.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:05:07.655 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:04:55.919+0000 7f0e89e47780 0 load: jerasure load: lrc 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:05:07.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:05:07.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:05:07.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:05:07.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:05:07.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:05:08.313 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:05:08.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:05:09.323 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:05:09.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:05:09.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:05:09.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:05:09.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:05:09.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:05:09.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:05:09.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:05:09.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:05:09.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:4' 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:09.634 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:05:09.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-13T19:05:09.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-13T19:05:09.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-13T19:05:09.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:09.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:05:09.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-13T19:05:09.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-13T19:05:09.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557' 2026-02-13T19:05:09.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:09.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:05:09.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444036 2026-02-13T19:05:09.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444036 2026-02-13T19:05:09.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036' 2026-02-13T19:05:09.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:09.825 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:05:09.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-13T19:05:09.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-13T19:05:09.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036 3-107374182403' 2026-02-13T19:05:09.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:09.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:05:09.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920771 2026-02-13T19:05:09.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920771 2026-02-13T19:05:09.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036 3-107374182403 4-141733920771' 2026-02-13T19:05:09.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:09.954 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-13T19:05:09.954 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:09.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:05:09.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-13T19:05:09.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:09.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-13T19:05:09.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-13T19:05:09.957 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803781 2026-02-13T19:05:09.957 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:05:10.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-13T19:05:10.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:10.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607557 2026-02-13T19:05:10.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:10.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:05:10.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-13T19:05:10.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:10.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607557 2026-02-13T19:05:10.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-13T19:05:10.193 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607557 2026-02-13T19:05:10.193 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:05:10.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-13T19:05:10.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:10.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444036 2026-02-13T19:05:10.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:10.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:05:10.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444036 2026-02-13T19:05:10.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:10.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444036 2026-02-13T19:05:10.418 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 73014444036 2026-02-13T19:05:10.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-13T19:05:10.418 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:05:10.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-13T19:05:10.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:10.645 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182403 2026-02-13T19:05:10.645 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:10.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:05:10.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-13T19:05:10.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:10.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182403 2026-02-13T19:05:10.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-13T19:05:10.648 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182403 2026-02-13T19:05:10.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:05:10.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182401 -lt 107374182403 2026-02-13T19:05:10.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:05:11.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:05:11.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:05:12.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182404 -lt 107374182403 2026-02-13T19:05:12.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:12.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920771 2026-02-13T19:05:12.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:12.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:05:12.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920771 2026-02-13T19:05:12.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:12.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920771 2026-02-13T19:05:12.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920771' 2026-02-13T19:05:12.102 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 141733920771 2026-02-13T19:05:12.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:05:12.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920771 -lt 141733920771 2026-02-13T19:05:12.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:05:12.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:05:12.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:05:12.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:05:12.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:05:12.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:05:12.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:05:12.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:05:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:05:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:05:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:05:13.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-13T19:05:13.388 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:05:13.388 INFO:tasks.workunit.client.0.trial034.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:08.682526+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-13T19:05:08.250704+0000 0'0 2026-02-13T19:05:08.250704+0000 0 0 periodic scrub scheduled @ 2026-02-15T02:16:42.632532+0000 0 0 2026-02-13T19:05:13.388 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456011+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:33:37.349532+0000 0 0 2026-02-13T19:05:13.388 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456510+0000 0'0 41:95 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:12:17.684736+0000 0 0 2026-02-13T19:05:13.389 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.665735+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T05:42:13.188691+0000 0 0 2026-02-13T19:05:13.389 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:01.862440+0000 0'0 41:63 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T04:40:25.959773+0000 0 0 2026-02-13T19:05:13.389 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:05:13.389 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:05:13.389 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:05:13.397 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:05:13.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:05:13.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:05:13.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:05:13.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:13.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-13T19:05:13.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-13T19:05:13.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-13T19:05:13.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:05:13.415 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T19:05:13.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-13T19:05:13.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-13T19:05:13.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:05:13.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:05:13.639 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:05:13.639 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-13T19:05:13.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:05:13.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:05:13.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:05:13.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-13T19:05:14.193 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already down. 2026-02-13T19:05:14.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-13T19:05:14.480 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already out. 2026-02-13T19:05:14.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-13T19:05:14.705 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:08.682526+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-13T19:05:08.250704+0000 0'0 2026-02-13T19:05:08.250704+0000 0 0 periodic scrub scheduled @ 2026-02-15T02:16:42.632532+0000 0 0 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456011+0000 0'0 41:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:33:37.349532+0000 0 0 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456510+0000 0'0 41:95 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:12:17.684736+0000 0 0 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.665735+0000 0'0 41:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T05:42:13.188691+0000 0 0 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:01.862440+0000 0'0 41:63 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T04:40:25.959773+0000 0 0 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:05:14.706 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:05:14.707 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:05:14.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-13T19:05:14.713 INFO:tasks.workunit.client.0.trial034.stderr:4+0 records in 2026-02-13T19:05:14.714 INFO:tasks.workunit.client.0.trial034.stderr:4+0 records out 2026-02-13T19:05:14.714 INFO:tasks.workunit.client.0.trial034.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.1132e-05 s, 57.6 MB/s 2026-02-13T19:05:14.714 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-13T19:05:14.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:14.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:14.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.222 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:15.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:15.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.855 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.855 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:16.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:16.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:17.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:17.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.047 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:18.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:18.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:19.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:05:19.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-13T19:05:19.696 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:05:19.925 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:05:19.925 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:2147483647' 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:05:19.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:05:19.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:05:19.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:05:19.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:05:19.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:05:19.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:05:19.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:05:19.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:05:19.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:05:19.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:05:19.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:05:19.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:05:19.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:05:20.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-13T19:05:20.024 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:05:20.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:05:20.256 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:05:20.256 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:05:20.256 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:05:20.256 INFO:tasks.workunit.client.0.trial034.stderr:2147483647' 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-13T19:05:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-13T19:05:20.258 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:05:20.259 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:05:20.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:05:20.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:05:20.306 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:05:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-13T19:05:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-13T19:05:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:05:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:05:20.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:05:20.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:05:20.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:05:20.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:05:20.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:05:20.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:05:20.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:05:20.359 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:05:20.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:05:20.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:05:20.360 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:05:20.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:05:20.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:05:20.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:05:20.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:05:20.370 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:20.368+0000 7f14b6e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:20.371 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:20.370+0000 7f14b6e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:20.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:20.370+0000 7f14b6e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:20.508 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:20.505+0000 7f14b6e3a780 -1 Falling back to public interface 2026-02-13T19:05:20.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:05:20.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:05:20.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:05:20.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:05:20.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:05:20.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:20.583 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:05:20.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:05:20.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:20.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:20.804 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:05:20.876 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:20.873+0000 7f14b6e3a780 -1 osd.2 41 log_to_monitors true 2026-02-13T19:05:21.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:05:21.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:21.805 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:05:21.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:05:21.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:21.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:22.033 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up out weight 0 up_from 47 up_thru 17 down_at 42 last_clean_interval [17,41) [v2:127.0.0.1:6818/2924813270,v1:127.0.0.1:6819/2924813270] [v2:127.0.0.1:6820/2924813270,v1:127.0.0.1:6821/2924813270] exists,up 509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:05:22.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:05:22.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:05:22.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:05:22.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-13T19:05:22.315 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already in. 2026-02-13T19:05:22.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-13T19:05:37.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-13T19:05:37.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-13T19:05:37.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-13T19:05:37.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-13T19:05:37.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-13T19:05:37.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-13T19:05:37.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-13T19:05:37.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-13T19:05:37.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-13T19:05:37.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-13T19:05:37.551 INFO:tasks.workunit.client.0.trial034.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-13T19:05:37.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-13T19:05:37.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-13T19:05:37.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-13T19:05:37.764 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-13T19:05:22.927371+0000 45'301 53:908 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-13T19:05:08.250704+0000 0'0 2026-02-13T19:05:08.250704+0000 0 0 no scrub is scheduled 0 0 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:22.347449+0000 0'0 53:63 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T00:04:36.668157+0000 0 0 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456510+0000 0'0 53:118 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:12:17.684736+0000 0 0 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.665735+0000 0'0 53:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T05:42:13.188691+0000 0 0 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:22.347590+0000 0'0 53:103 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T03:13:18.064671+0000 0 0 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:05:37.765 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:05:37.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:05:37.771 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:05:37.772 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:05:37.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:05:37.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:05:37.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:05:38.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:05:38.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-13T19:05:43.080 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-13T19:05:43.293 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:05:43.293 INFO:tasks.workunit.client.0.trial034.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-13T19:05:37.930970+0000 45'301 55:920 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-13T19:05:08.250704+0000 0'0 2026-02-13T19:05:08.250704+0000 0 0 no scrub is scheduled 0 0 2026-02-13T19:05:43.293 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:22.347449+0000 0'0 53:63 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T00:04:36.668157+0000 0 0 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.456510+0000 0'0 53:118 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:12:17.684736+0000 0 0 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:05:07.665735+0000 0'0 53:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T05:42:13.188691+0000 0 0 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-13T19:05:37.929024+0000 0'0 55:111 [1,0] 1 [1,0] 1 0'0 2026-02-13T19:04:53.955831+0000 0'0 2026-02-13T19:04:53.955831+0000 0 0 periodic scrub scheduled @ 2026-02-15T06:08:07.547108+0000 0 0 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:05:43.294 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:05:43.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout:{ 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: "num_missing": 1, 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: "num_unfound": 1, 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: "objects": [ 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: "oid": { 2026-02-13T19:05:43.362 INFO:tasks.workunit.client.0.trial034.stdout: "oid": "obj250", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "key": "", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "snapid": -2, 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "hash": 2249616407, 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "max": 0, 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "pool": 2, 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "namespace": "" 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "need": "45'251", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "have": "0'0", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "flags": "none", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-13T19:05:43.363 INFO:tasks.workunit.client.0.trial034.stdout: "locations": [ 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: ] 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "state": "NotRecovering", 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "available_might_have_unfound": true, 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "might_have_unfound": [ 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "2(4)", 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "status": "osd is down" 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.364 INFO:tasks.workunit.client.0.trial034.stdout: "more": false 2026-02-13T19:05:43.365 INFO:tasks.workunit.client.0.trial034.stdout:} 2026-02-13T19:05:43.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-13T19:05:43.428 INFO:tasks.workunit.client.0.trial034.stdout:{ 2026-02-13T19:05:43.428 INFO:tasks.workunit.client.0.trial034.stdout: "snap_trimq": "[]", 2026-02-13T19:05:43.428 INFO:tasks.workunit.client.0.trial034.stdout: "snap_trimq_len": 0, 2026-02-13T19:05:43.428 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: "epoch": 55, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.429 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "acting_recovery_backfill": [ 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "info": { 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:05:43.430 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "0", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'301", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'250", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "45'200", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 301, 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "54", 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.431 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 54, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 54, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 54, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.432 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'301", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 921, 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 55, 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:05:37.930970+0000", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:05:37.930970+0000", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:05:37.930970+0000", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:05:37.930970+0000", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:05:13.412425+0000", 2026-02-13T19:05:43.433 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:05:37.929337+0000", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:05:37.929337+0000", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:05:37.930970+0000", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:05:37.928505+0000", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:05:37.928405+0000", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 54, 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "45'200", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "45'200", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:05:43.434 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 200, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:05:43.435 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 1232896, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 301, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:05:43.436 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 1505, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 1, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 1, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 303, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 1, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 301, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 301, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 1204, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:05:43.437 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:05:43.438 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.439 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:05:43.440 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 300 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),4(2)", 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 1 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.441 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.442 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "peer_info": [ 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "0(3)", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s3", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "3", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'301", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'301", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "45'200", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 301, 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:05:43.443 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 54, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 54, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 54, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:05:43.444 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'301", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 911, 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 54, 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.445 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:05:13.412425+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:05:14.025771+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:05:14.025678+0000", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 54, 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "45'200", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "45'200", 2026-02-13T19:05:43.446 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 200, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:05:43.447 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-15T06:54:06.744092+0000", 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:05:43.448 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 1232896, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 301, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 1505, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 301, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 301, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:05:43.449 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 301, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 1204, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:05:43.450 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:05:43.451 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:05:43.452 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:05:43.453 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 301 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:05:43.454 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "1(1)", 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s1", 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "1", 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'301", 2026-02-13T19:05:43.455 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'301", 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "45'200", 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 301, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 54, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.456 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 54, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 54, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.457 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'301", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 332, 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 45, 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.458 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:05:13.412425+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:05:14.025771+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:05:14.025678+0000", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 54, 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "45'200", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "45'200", 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.459 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 200, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:05:43.460 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-15T06:54:06.744092+0000", 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 1232896, 2026-02-13T19:05:43.461 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 301, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 1505, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 1, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 301, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 301, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 301, 2026-02-13T19:05:43.462 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 1204, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:05:43.463 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:05:43.464 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:05:43.465 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 301 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:05:43.466 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "4(2)", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s2", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "2", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'301", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'301", 2026-02-13T19:05:43.467 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "45'200", 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 301, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 54, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 54, 2026-02-13T19:05:43.468 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 54, 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'301", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 332, 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 45, 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.469 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:05:13.412425+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:05:14.026974+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:05:19.692579+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:05:14.025771+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:05:14.025678+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 54, 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "45'200", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "45'200", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.470 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:05:08.250704+0000", 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 200, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-15T06:54:06.744092+0000", 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:05:43.471 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 1232896, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 301, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 1505, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 301, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 301, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 301, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 1204, 2026-02-13T19:05:43.472 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:05:43.473 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:05:43.474 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: 2147483647 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 301 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:05:43.475 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 55, 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_state": [ 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "name": "Started/Primary/Active", 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "enter_time": "2026-02-13T19:05:37.928421+0000", 2026-02-13T19:05:43.476 INFO:tasks.workunit.client.0.trial034.stdout: "might_have_unfound": [ 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "0(3)", 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "1(1)", 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "2(4)", 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "status": "osd is down" 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "4(2)", 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_progress": { 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "backfill_targets": [], 2026-02-13T19:05:43.477 INFO:tasks.workunit.client.0.trial034.stdout: "waiting_on_backfill": [], 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill_started": "MIN", 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "backfill_info": { 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "begin": "MIN", 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "end": "MIN", 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "objects": [] 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "peer_backfill_info": [], 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "backfills_in_flight": [], 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "recovering": [], 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "pg_backend": { 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_ops": [], 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "read_ops": [] 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "name": "Started", 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: "enter_time": "2026-02-13T19:05:37.828179+0000" 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.478 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "scrubber": { 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "active": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "must_scrub": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "must_deep_scrub": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "must_repair": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-15T02:58:52.786643+0000", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "schedule": "not registered for scrubbing", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "shallow-target": { 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "level": "shallow", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-15T02:58:52.786643+0000", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "orig_sched_time": "2026-02-15T02:58:52.786643+0000", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "last_issue": "ok", 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "forced": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "eligible": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "queued": false, 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: "active": false 2026-02-13T19:05:43.479 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "deep-target": { 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "level": "deep", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-19T01:08:39.281826+0000", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "orig_sched_time": "2026-02-19T01:08:39.281826+0000", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "last_issue": "ok", 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "forced": false, 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "eligible": false, 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "queued": false, 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "active": false 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout: "agent_state": {} 2026-02-13T19:05:43.480 INFO:tasks.workunit.client.0.trial034.stdout:} 2026-02-13T19:05:43.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:05:43.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-13T19:05:43.498 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:05:43.498 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-13T19:05:43.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-13T19:05:43.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-13T19:05:43.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:05:43.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-13T19:05:43.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-13T19:05:43.627 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-13T19:05:43.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-13T19:05:43.628 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:05:43.628 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-13T19:05:43.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:05:43.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:43.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:05:43.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:05:43.700 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:05:43.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:05:43.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:05:43.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:05:43.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:05:43.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:05:43.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:05:43.711 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:43.710+0000 7ff0cea68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:43.712 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:43.711+0000 7ff0cea68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:43.713 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:43.712+0000 7ff0cea68780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:05:43.848 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:43.847+0000 7ff0cea68780 -1 Falling back to public interface 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:05:43.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:43.931 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:05:43.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:05:43.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:43.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:44.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:05:44.258 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:05:44.257+0000 7ff0cea68780 -1 osd.2 53 log_to_monitors true 2026-02-13T19:05:45.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:05:45.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:05:45.160 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:05:45.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:05:45.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:05:45.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:05:45.385 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 57 up_thru 17 down_at 54 last_clean_interval [47,53) [v2:127.0.0.1:6818/2549042406,v1:127.0.0.1:6819/2549042406] [v2:127.0.0.1:6820/2549042406,v1:127.0.0.1:6821/2549042406] exists,up 509c9de8-03b1-40a7-8886-f5644db5d972 2026-02-13T19:05:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:05:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:05:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:05:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-13T19:05:50.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-13T19:05:50.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-13T19:05:50.454 INFO:tasks.workunit.client.0.trial034.stderr:pg has no unfound objects 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:05:50.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:05:50.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:05:50.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:05:50.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:4' 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:50.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:05:50.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-13T19:05:50.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-13T19:05:50.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-13T19:05:50.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:50.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:05:50.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607567 2026-02-13T19:05:50.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607567 2026-02-13T19:05:50.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607567' 2026-02-13T19:05:50.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:50.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:05:50.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135876 2026-02-13T19:05:50.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135876 2026-02-13T19:05:50.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-244813135876' 2026-02-13T19:05:50.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:50.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:05:51.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182414 2026-02-13T19:05:51.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182414 2026-02-13T19:05:51.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-244813135876 3-107374182414' 2026-02-13T19:05:51.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:05:51.036 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:05:51.101 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920781 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920781 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-244813135876 3-107374182414 4-141733920781' 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:51.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:05:51.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-13T19:05:51.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:51.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-13T19:05:51.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-13T19:05:51.105 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803791 2026-02-13T19:05:51.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:05:51.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-13T19:05:51.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:05:52.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:05:52.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:05:52.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-13T19:05:52.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:52.557 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607567 2026-02-13T19:05:52.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:52.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:05:52.559 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607567 2026-02-13T19:05:52.559 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:52.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607567 2026-02-13T19:05:52.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607567' 2026-02-13T19:05:52.561 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607567 2026-02-13T19:05:52.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:05:52.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607567 -lt 51539607567 2026-02-13T19:05:52.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:52.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-244813135876 2026-02-13T19:05:52.786 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:52.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:05:52.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-244813135876 2026-02-13T19:05:52.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:52.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=244813135876 2026-02-13T19:05:52.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 244813135876' 2026-02-13T19:05:52.789 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 244813135876 2026-02-13T19:05:52.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:05:53.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 244813135876 -lt 244813135876 2026-02-13T19:05:53.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:53.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182414 2026-02-13T19:05:53.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:53.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:05:53.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182414 2026-02-13T19:05:53.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:53.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182414 2026-02-13T19:05:53.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182414' 2026-02-13T19:05:53.021 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182414 2026-02-13T19:05:53.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:05:53.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182414 -lt 107374182414 2026-02-13T19:05:53.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:05:53.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920781 2026-02-13T19:05:53.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:05:53.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:05:53.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920781 2026-02-13T19:05:53.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:05:53.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920781 2026-02-13T19:05:53.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920781' 2026-02-13T19:05:53.248 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 141733920781 2026-02-13T19:05:53.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:05:53.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920781 -lt 141733920781 2026-02-13T19:05:53.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:05:53.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:05:53.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:05:53.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:05:54.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:05:54.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:05:54.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:05:54.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:05:54.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:05:54.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:05:54.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:05:54.350 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-13T19:05:54.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-13T19:05:54.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-13T19:05:54.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-13T19:05:54.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-13T19:05:54.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-13T19:05:54.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-13T19:05:54.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-13T19:05:54.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-13T19:05:54.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-13T19:05:54.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-13T19:05:54.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-13T19:05:54.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-13T19:05:54.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-13T19:05:54.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-13T19:05:54.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-13T19:05:54.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-13T19:05:54.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-13T19:05:54.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-13T19:05:54.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-13T19:05:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-13T19:05:54.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-13T19:05:54.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-13T19:05:54.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-13T19:05:54.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-13T19:05:54.739 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-13T19:05:54.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-13T19:05:54.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-13T19:05:54.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-13T19:05:54.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-13T19:05:54.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-13T19:05:54.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.856 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-13T19:05:54.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-13T19:05:54.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-13T19:05:54.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-13T19:05:54.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-13T19:05:54.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-13T19:05:54.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-13T19:05:54.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-13T19:05:54.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-13T19:05:54.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:54.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:54.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-13T19:05:54.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-13T19:05:55.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.024 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-13T19:05:55.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-13T19:05:55.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-13T19:05:55.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-13T19:05:55.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-13T19:05:55.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-13T19:05:55.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.124 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-13T19:05:55.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-13T19:05:55.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-13T19:05:55.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-13T19:05:55.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-13T19:05:55.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-13T19:05:55.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-13T19:05:55.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-13T19:05:55.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-13T19:05:55.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-13T19:05:55.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-13T19:05:55.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-13T19:05:55.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-13T19:05:55.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-13T19:05:55.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-13T19:05:55.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-13T19:05:55.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-13T19:05:55.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-13T19:05:55.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-13T19:05:55.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-13T19:05:55.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-13T19:05:55.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-13T19:05:55.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-13T19:05:55.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-13T19:05:55.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-13T19:05:55.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-13T19:05:55.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.561 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-13T19:05:55.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-13T19:05:55.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-13T19:05:55.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-13T19:05:55.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-13T19:05:55.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-13T19:05:55.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-13T19:05:55.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-13T19:05:55.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-13T19:05:55.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-13T19:05:55.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-13T19:05:55.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-13T19:05:55.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-13T19:05:55.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-13T19:05:55.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-13T19:05:55.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-13T19:05:55.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-13T19:05:55.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-13T19:05:55.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-13T19:05:55.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-13T19:05:55.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-13T19:05:55.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-13T19:05:55.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-13T19:05:55.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-13T19:05:55.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-13T19:05:55.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-13T19:05:55.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-13T19:05:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.002 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-13T19:05:56.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-13T19:05:56.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-13T19:05:56.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-13T19:05:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-13T19:05:56.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-13T19:05:56.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-13T19:05:56.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-13T19:05:56.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-13T19:05:56.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-13T19:05:56.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-13T19:05:56.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-13T19:05:56.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-13T19:05:56.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.222 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-13T19:05:56.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-13T19:05:56.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-13T19:05:56.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-13T19:05:56.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-13T19:05:56.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.306 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.306 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-13T19:05:56.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-13T19:05:56.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-13T19:05:56.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-13T19:05:56.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-13T19:05:56.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-13T19:05:56.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-13T19:05:56.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-13T19:05:56.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-13T19:05:56.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-13T19:05:56.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-13T19:05:56.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.490 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-13T19:05:56.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-13T19:05:56.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.524 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-13T19:05:56.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-13T19:05:56.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.559 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.559 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-13T19:05:56.559 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-13T19:05:56.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-13T19:05:56.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-13T19:05:56.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-13T19:05:56.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-13T19:05:56.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-13T19:05:56.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-13T19:05:56.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-13T19:05:56.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-13T19:05:56.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-13T19:05:56.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-13T19:05:56.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-13T19:05:56.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.775 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-13T19:05:56.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-13T19:05:56.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-13T19:05:56.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-13T19:05:56.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-13T19:05:56.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-13T19:05:56.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.874 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-13T19:05:56.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-13T19:05:56.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-13T19:05:56.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-13T19:05:56.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-13T19:05:56.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-13T19:05:56.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-13T19:05:56.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-13T19:05:56.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:56.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:56.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-13T19:05:56.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-13T19:05:57.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-13T19:05:57.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-13T19:05:57.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-13T19:05:57.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-13T19:05:57.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-13T19:05:57.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-13T19:05:57.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-13T19:05:57.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-13T19:05:57.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-13T19:05:57.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-13T19:05:57.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-13T19:05:57.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-13T19:05:57.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-13T19:05:57.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-13T19:05:57.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-13T19:05:57.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-13T19:05:57.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-13T19:05:57.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-13T19:05:57.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-13T19:05:57.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-13T19:05:57.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-13T19:05:57.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-13T19:05:57.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-13T19:05:57.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-13T19:05:57.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-13T19:05:57.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-13T19:05:57.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-13T19:05:57.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-13T19:05:57.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-13T19:05:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-13T19:05:57.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-13T19:05:57.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-13T19:05:57.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-13T19:05:57.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-13T19:05:57.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-13T19:05:57.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-13T19:05:57.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-13T19:05:57.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-13T19:05:57.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-13T19:05:57.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-13T19:05:57.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-13T19:05:57.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-13T19:05:57.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-13T19:05:57.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-13T19:05:57.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-13T19:05:57.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-13T19:05:57.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-13T19:05:57.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-13T19:05:57.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-13T19:05:57.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-13T19:05:57.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-13T19:05:57.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-13T19:05:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-13T19:05:57.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-13T19:05:57.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-13T19:05:57.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-13T19:05:57.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-13T19:05:57.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-13T19:05:57.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:57.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:57.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-13T19:05:57.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-13T19:05:58.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-13T19:05:58.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-13T19:05:58.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-13T19:05:58.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-13T19:05:58.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-13T19:05:58.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-13T19:05:58.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-13T19:05:58.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-13T19:05:58.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-13T19:05:58.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-13T19:05:58.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-13T19:05:58.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-13T19:05:58.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-13T19:05:58.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-13T19:05:58.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-13T19:05:58.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-13T19:05:58.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-13T19:05:58.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-13T19:05:58.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-13T19:05:58.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-13T19:05:58.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-13T19:05:58.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-13T19:05:58.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-13T19:05:58.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-13T19:05:58.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-13T19:05:58.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-13T19:05:58.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-13T19:05:58.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-13T19:05:58.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-13T19:05:58.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-13T19:05:58.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-13T19:05:58.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-13T19:05:58.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.559 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-13T19:05:58.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-13T19:05:58.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-13T19:05:58.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-13T19:05:58.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-13T19:05:58.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.627 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-13T19:05:58.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-13T19:05:58.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-13T19:05:58.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-13T19:05:58.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-13T19:05:58.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-13T19:05:58.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-13T19:05:58.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-13T19:05:58.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-13T19:05:58.772 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-13T19:05:58.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-13T19:05:58.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-13T19:05:58.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-13T19:05:58.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.856 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-13T19:05:58.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.874 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-13T19:05:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-13T19:05:58.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-13T19:05:58.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-13T19:05:58.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-13T19:05:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-13T19:05:58.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-13T19:05:58.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-13T19:05:58.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:58.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:58.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-13T19:05:58.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-13T19:05:59.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-13T19:05:59.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-13T19:05:59.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-13T19:05:59.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-13T19:05:59.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-13T19:05:59.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-13T19:05:59.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-13T19:05:59.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-13T19:05:59.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-13T19:05:59.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-13T19:05:59.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-13T19:05:59.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-13T19:05:59.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-13T19:05:59.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-13T19:05:59.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-13T19:05:59.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-13T19:05:59.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-13T19:05:59.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-13T19:05:59.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-13T19:05:59.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-13T19:05:59.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-13T19:05:59.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-13T19:05:59.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:05:59.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-13T19:05:59.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-13T19:05:59.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:05:59.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:05:59.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:05:59.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:05:59.688 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:05:59.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:05:59.983 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:05:59.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:05:59.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:05:59.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:05:59.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:05:59.993 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:05:59.993 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:05:59.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:05:59.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:05:59.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:06:00.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:06:00.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:06:00.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:06:00.104 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:06:00.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:06:00.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:06:00.105 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:06:00.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:00.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:06:00.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:06:00.107 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:00.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:06:00.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:06:00.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:06:00.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:06:00.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:06:00.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:06:00.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:06:00.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:06:00.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:06:00.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:06:00.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:06:00.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:06:00.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:06:00.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:06:00.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:06:00.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:06:00.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:06:00.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:00.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:06:00.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:06:00.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:00.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:06:00.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:06:00.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:06:00.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:06:00.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:06:00.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:06:00.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:06:00.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:06:00.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:06:00.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:06:00.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:06:00.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:06:00.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:06:00.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:00.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:00.178 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:00.178 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.178 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:00.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:06:00.200 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:06:00.201 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:06:00.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:06:00.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:06:00.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:06:00.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:06:00.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:06:00.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:06:00.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:06:00.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:06:00.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:06:00.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:06:00.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:06:00.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:06:00.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:06:00.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:00.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:00.387 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:00.387 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:00.388 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:00.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:00.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:06:00.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:06:00.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:06:00.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:06:00.499 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:06:00.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:06:01.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:06:01.547 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:06:01.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:06:00.194+0000 7fe6f44fdc00 0 load: jerasure load: lrc 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:06:01.553 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-13T19:06:01.554 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:01.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:01.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=596ff38c-1b67-4205-9498-2f7dfdb429d3 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 596ff38c-1b67-4205-9498-2f7dfdb429d3' 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 596ff38c-1b67-4205-9498-2f7dfdb429d3 2026-02-13T19:06:01.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:01.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZdo9pr4PDIRAAu8tzaXST5zfzFryjbV2a4A== 2026-02-13T19:06:01.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZdo9pr4PDIRAAu8tzaXST5zfzFryjbV2a4A=="}' 2026-02-13T19:06:01.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 596ff38c-1b67-4205-9498-2f7dfdb429d3 -i td/test-erasure-eio/0/new.json 2026-02-13T19:06:01.666 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:01.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:06:01.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZdo9pr4PDIRAAu8tzaXST5zfzFryjbV2a4A== --osd-uuid 596ff38c-1b67-4205-9498-2f7dfdb429d3 2026-02-13T19:06:01.689 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:01.689+0000 7f12e0052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:01.691 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:01.690+0000 7f12e0052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:01.691 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:01.691+0000 7f12e0052780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:01.692 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:01.691+0000 7f12e0052780 -1 bdev(0x3c32c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:01.692 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:01.691+0000 7f12e0052780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:06:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:06:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:02.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:06:02.114 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:06:02.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:02.217 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:06:02.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:06:02.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:02.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:02.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:02.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:02.232 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:02.231+0000 7f4a9760c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:02.233 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:02.233+0000 7f4a9760c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:02.234 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:02.233+0000 7f4a9760c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:02.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:02.316 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:02.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:02.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:06:02.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:02.455 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:02.454+0000 7f4a9760c780 -1 Falling back to public interface 2026-02-13T19:06:02.710 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:02.708+0000 7f4a9760c780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:06:03.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:03.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:03.414 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:03.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:03.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:03.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:06:03.646 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2091867479,v1:127.0.0.1:6803/2091867479] [v2:127.0.0.1:6804/2091867479,v1:127.0.0.1:6805/2091867479] exists,up 596ff38c-1b67-4205-9498-2f7dfdb429d3 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:03.647 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:06:03.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:03.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=38737781-e2b0-42b5-8bb5-c2488ae431f0 2026-02-13T19:06:03.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 38737781-e2b0-42b5-8bb5-c2488ae431f0' 2026-02-13T19:06:03.649 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 38737781-e2b0-42b5-8bb5-c2488ae431f0 2026-02-13T19:06:03.649 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:03.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAbdo9pauBGJxAAH1BcUCDSrjxrPQMBq1mRHA== 2026-02-13T19:06:03.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAbdo9pauBGJxAAH1BcUCDSrjxrPQMBq1mRHA=="}' 2026-02-13T19:06:03.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 38737781-e2b0-42b5-8bb5-c2488ae431f0 -i td/test-erasure-eio/1/new.json 2026-02-13T19:06:03.886 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:03.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:06:03.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAbdo9pauBGJxAAH1BcUCDSrjxrPQMBq1mRHA== --osd-uuid 38737781-e2b0-42b5-8bb5-c2488ae431f0 2026-02-13T19:06:03.911 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:03.909+0000 7f61d1653780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:03.912 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:03.911+0000 7f61d1653780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:03.913 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:03.912+0000 7f61d1653780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:03.913 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:03.912+0000 7f61d1653780 -1 bdev(0x3980c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:03.914 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:03.912+0000 7f61d1653780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:06:04.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:06:04.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:04.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:06:04.239 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:06:04.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:04.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:06:04.559 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:06:04.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:04.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:04.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:04.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:04.573 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:04.571+0000 7fec02a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:04.574 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:04.573+0000 7fec02a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:04.575 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:04.574+0000 7fec02a13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:04.734 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:04.731+0000 7fec02a13780 -1 Falling back to public interface 2026-02-13T19:06:04.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:06:04.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:04.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:06:05.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:05.037 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:05.036+0000 7fec02a13780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:06:06.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:06.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:06.008 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:06.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:06.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:06.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:06:06.232 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3417481378,v1:127.0.0.1:6811/3417481378] [v2:127.0.0.1:6812/3417481378,v1:127.0.0.1:6813/3417481378] exists,up 38737781-e2b0-42b5-8bb5-c2488ae431f0 2026-02-13T19:06:06.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:06.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:06.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:06.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:06.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:06.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:06:06.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:06.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=08921e45-58bf-4228-a504-0594d584a2d9 2026-02-13T19:06:06.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 08921e45-58bf-4228-a504-0594d584a2d9' 2026-02-13T19:06:06.240 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 08921e45-58bf-4228-a504-0594d584a2d9 2026-02-13T19:06:06.241 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:06.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAedo9pvqaZDhAAtH8cWNX0RHKaoxVGG0JtbQ== 2026-02-13T19:06:06.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAedo9pvqaZDhAAtH8cWNX0RHKaoxVGG0JtbQ=="}' 2026-02-13T19:06:06.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 08921e45-58bf-4228-a504-0594d584a2d9 -i td/test-erasure-eio/2/new.json 2026-02-13T19:06:06.476 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:06:06.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:06:06.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAedo9pvqaZDhAAtH8cWNX0RHKaoxVGG0JtbQ== --osd-uuid 08921e45-58bf-4228-a504-0594d584a2d9 2026-02-13T19:06:06.501 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:06.498+0000 7f8187a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:06.502 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:06.500+0000 7f8187a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:06.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:06.501+0000 7f8187a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:06.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:06.501+0000 7f8187a29780 -1 bdev(0x3c6cc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:06.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:06.501+0000 7f8187a29780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:06:06.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:06:06.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:06.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:06:06.820 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:06:06.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:07.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:07.153 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:07.151+0000 7f057f1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:07.154 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:07.153+0000 7f057f1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:07.155 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:07.154+0000 7f057f1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:07.284 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:07.283+0000 7f057f1d8780 -1 Falling back to public interface 2026-02-13T19:06:07.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:07.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:06:07.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:07.632 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:07.630+0000 7f057f1d8780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:06:08.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:08.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:08.587 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:08.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:08.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:08.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:06:08.813 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3838651889,v1:127.0.0.1:6819/3838651889] [v2:127.0.0.1:6820/3838651889,v1:127.0.0.1:6821/3838651889] exists,up 08921e45-58bf-4228-a504-0594d584a2d9 2026-02-13T19:06:08.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:08.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:08.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:08.819 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:08.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5e295671-cd7d-4e05-a525-61d3fdccbdfc 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 5e295671-cd7d-4e05-a525-61d3fdccbdfc' 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 5e295671-cd7d-4e05-a525-61d3fdccbdfc 2026-02-13T19:06:08.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:08.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgdo9p8l9AMRAAbt2SfOcd++3jPUR8ZNdvUg== 2026-02-13T19:06:08.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgdo9p8l9AMRAAbt2SfOcd++3jPUR8ZNdvUg=="}' 2026-02-13T19:06:08.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5e295671-cd7d-4e05-a525-61d3fdccbdfc -i td/test-erasure-eio/3/new.json 2026-02-13T19:06:09.057 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:06:09.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:06:09.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgdo9p8l9AMRAAbt2SfOcd++3jPUR8ZNdvUg== --osd-uuid 5e295671-cd7d-4e05-a525-61d3fdccbdfc 2026-02-13T19:06:09.082 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.081+0000 7f15d0444780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.084 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.082+0000 7f15d0444780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.085 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.083+0000 7f15d0444780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.085 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.083+0000 7f15d0444780 -1 bdev(0x286cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:09.085 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.083+0000 7f15d0444780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:06:09.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:06:09.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:09.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:06:09.415 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:06:09.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:09.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:06:09.753 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:06:09.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:09.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:09.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:09.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:09.768 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.766+0000 7f4e9f03a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.770 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.768+0000 7f4e9f03a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.770 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.769+0000 7f4e9f03a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:09.900 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:09.899+0000 7f4e9f03a780 -1 Falling back to public interface 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:09.980 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:09.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:09.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:09.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:10.204 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:10.202+0000 7f4e9f03a780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:06:10.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:11.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:11.207 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:11.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:11.443 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3823174617,v1:127.0.0.1:6827/3823174617] [v2:127.0.0.1:6828/3823174617,v1:127.0.0.1:6829/3823174617] exists,up 5e295671-cd7d-4e05-a525-61d3fdccbdfc 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:06:11.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:06:11.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:11.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:11.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=815c92fe-394a-4463-98dd-24639e5e54f9 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 815c92fe-394a-4463-98dd-24639e5e54f9' 2026-02-13T19:06:11.452 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 815c92fe-394a-4463-98dd-24639e5e54f9 2026-02-13T19:06:11.453 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:11.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAjdo9pwiwoGxAAOb6deaQjfiFExE+uzjUylA== 2026-02-13T19:06:11.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAjdo9pwiwoGxAAOb6deaQjfiFExE+uzjUylA=="}' 2026-02-13T19:06:11.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 815c92fe-394a-4463-98dd-24639e5e54f9 -i td/test-erasure-eio/4/new.json 2026-02-13T19:06:11.687 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:06:11.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:06:11.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAjdo9pwiwoGxAAOb6deaQjfiFExE+uzjUylA== --osd-uuid 815c92fe-394a-4463-98dd-24639e5e54f9 2026-02-13T19:06:11.712 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:11.710+0000 7fc3c7fd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:11.713 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:11.711+0000 7fc3c7fd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:11.714 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:11.712+0000 7fc3c7fd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:11.715 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:11.712+0000 7fc3c7fd6780 -1 bdev(0x264ec00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:11.715 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:11.713+0000 7fc3c7fd6780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:06:12.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:06:12.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:12.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:06:12.027 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:06:12.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:12.349 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:06:12.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:06:12.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:12.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:12.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:12.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:12.364 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:12.362+0000 7fec1cc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:12.365 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:12.364+0000 7fec1cc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:12.366 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:12.365+0000 7fec1cc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:12.518 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:12.516+0000 7fec1cc46780 -1 Falling back to public interface 2026-02-13T19:06:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:06:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:06:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:12.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:12.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:12.581 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:12.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:12.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:12.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:06:12.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:12.826 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:12.824+0000 7fec1cc46780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:06:13.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:13.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:13.809 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:13.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:13.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:13.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:06:14.049 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/480992852,v1:127.0.0.1:6835/480992852] [v2:127.0.0.1:6836/480992852,v1:127.0.0.1:6837/480992852] exists,up 815c92fe-394a-4463-98dd-24639e5e54f9 2026-02-13T19:06:14.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:14.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:14.055 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:14.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d4c05d00-98a5-4d6f-aa1b-d86057724427 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 d4c05d00-98a5-4d6f-aa1b-d86057724427' 2026-02-13T19:06:14.057 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 d4c05d00-98a5-4d6f-aa1b-d86057724427 2026-02-13T19:06:14.058 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:14.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAmdo9pWpKxAxAAjxtO0witaM857XT+40+3AA== 2026-02-13T19:06:14.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAmdo9pWpKxAxAAjxtO0witaM857XT+40+3AA=="}' 2026-02-13T19:06:14.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d4c05d00-98a5-4d6f-aa1b-d86057724427 -i td/test-erasure-eio/5/new.json 2026-02-13T19:06:14.292 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:06:14.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-13T19:06:14.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmdo9pWpKxAxAAjxtO0witaM857XT+40+3AA== --osd-uuid d4c05d00-98a5-4d6f-aa1b-d86057724427 2026-02-13T19:06:14.317 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.315+0000 7fcaab5d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:14.318 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.316+0000 7fcaab5d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:14.319 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.317+0000 7fcaab5d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:14.320 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.317+0000 7fcaab5d8780 -1 bdev(0x4176c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:14.320 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.318+0000 7fcaab5d8780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-13T19:06:14.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-13T19:06:14.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:14.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:06:14.653 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:06:14.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:14.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:06:14.966 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:06:14.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:14.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:14.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:14.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:14.981 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.979+0000 7ffb4b431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:14.983 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.981+0000 7ffb4b431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:14.983 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:14.982+0000 7ffb4b431780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:15.116 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:15.115+0000 7ffb4b431780 -1 Falling back to public interface 2026-02-13T19:06:15.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:06:15.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:15.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:06:15.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:15.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:15.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:15.193 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:15.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:15.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:15.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:06:15.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:15.443 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:15.441+0000 7ffb4b431780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:06:16.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:16.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:16.421 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:16.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:16.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:16.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:06:16.652 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3753739898,v1:127.0.0.1:6843/3753739898] [v2:127.0.0.1:6844/3753739898,v1:127.0.0.1:6845/3753739898] exists,up d4c05d00-98a5-4d6f-aa1b-d86057724427 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:16.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:16.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:16.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bd9f16e2-bfc0-496b-9425-e87d26c31b4b 2026-02-13T19:06:16.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 bd9f16e2-bfc0-496b-9425-e87d26c31b4b' 2026-02-13T19:06:16.660 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 bd9f16e2-bfc0-496b-9425-e87d26c31b4b 2026-02-13T19:06:16.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:16.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAodo9p4L6eJxAA11B7R0ZWefVcbV17kceMjQ== 2026-02-13T19:06:16.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAodo9p4L6eJxAA11B7R0ZWefVcbV17kceMjQ=="}' 2026-02-13T19:06:16.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bd9f16e2-bfc0-496b-9425-e87d26c31b4b -i td/test-erasure-eio/6/new.json 2026-02-13T19:06:16.900 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:06:16.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-13T19:06:16.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAodo9p4L6eJxAA11B7R0ZWefVcbV17kceMjQ== --osd-uuid bd9f16e2-bfc0-496b-9425-e87d26c31b4b 2026-02-13T19:06:16.924 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:16.922+0000 7f413b815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:16.925 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:16.923+0000 7f413b815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:16.926 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:16.924+0000 7f413b815780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:16.927 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:16.925+0000 7f413b815780 -1 bdev(0x37d4c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:16.927 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:16.925+0000 7f413b815780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-13T19:06:17.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-13T19:06:17.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:17.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:06:17.263 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:06:17.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:17.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:06:17.581 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:06:17.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:17.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:17.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:17.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:17.596 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:17.594+0000 7fe623c4f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:17.597 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:17.595+0000 7fe623c4f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:17.598 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:17.596+0000 7fe623c4f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:17.725 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:17.723+0000 7fe623c4f780 -1 Falling back to public interface 2026-02-13T19:06:17.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:17.807 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:17.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:17.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:06:18.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:18.049 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:18.047+0000 7fe623c4f780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:06:19.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:19.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:19.036 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:19.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:19.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:19.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:06:19.261 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/930244164,v1:127.0.0.1:6851/930244164] [v2:127.0.0.1:6852/930244164,v1:127.0.0.1:6853/930244164] exists,up bd9f16e2-bfc0-496b-9425-e87d26c31b4b 2026-02-13T19:06:19.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:19.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:19.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:06:19.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:06:19.306 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:06:19.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:06:19.312 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:06:02.456+0000 7f4a9760c780 0 load: jerasure load: lrc 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:06:19.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:06:19.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:06:19.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:06:20.012 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:06:20.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:06:21.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:06:21.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:06:21.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:06:21.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:06:21.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:06:21.023 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:06:21.023 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:06:21.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:06:21.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:06:21.024 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:06:21.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:06:21.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:06:21.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:06:21.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:06:21.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:06:21.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:06:21.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:06:21.344 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:06:21.344 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:06:21.344 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:06:21.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-13T19:06:21.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-13T19:06:21.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-13T19:06:21.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.407 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:06:21.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-13T19:06:21.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-13T19:06:21.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-13T19:06:21.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:06:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-13T19:06:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-13T19:06:21.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334' 2026-02-13T19:06:21.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.534 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:06:21.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-13T19:06:21.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-13T19:06:21.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-111669149701' 2026-02-13T19:06:21.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:06:21.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-13T19:06:21.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-13T19:06:21.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-111669149701 4-150323855364' 2026-02-13T19:06:21.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.663 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:06:21.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-13T19:06:21.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-13T19:06:21.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-111669149701 4-150323855364 5-188978561028' 2026-02-13T19:06:21.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:21.727 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:06:21.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-13T19:06:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-13T19:06:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-111669149701 4-150323855364 5-188978561028 6-223338299395' 2026-02-13T19:06:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:21.793 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-13T19:06:21.793 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:21.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:06:21.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-13T19:06:21.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:21.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-13T19:06:21.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-13T19:06:21.796 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803782 2026-02-13T19:06:21.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:06:22.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-13T19:06:22.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:06:23.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:06:23.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:06:23.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-13T19:06:23.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:23.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-13T19:06:23.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:23.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:06:23.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-13T19:06:23.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:23.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-13T19:06:23.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-13T19:06:23.259 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574854 2026-02-13T19:06:23.259 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:06:23.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-13T19:06:23.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:23.488 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411334 2026-02-13T19:06:23.489 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:23.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:06:23.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-13T19:06:23.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:23.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-13T19:06:23.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-13T19:06:23.492 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411334 2026-02-13T19:06:23.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:06:23.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-13T19:06:23.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:23.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-13T19:06:23.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:23.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:06:23.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-13T19:06:23.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:23.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-13T19:06:23.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-13T19:06:23.718 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149701 2026-02-13T19:06:23.718 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:06:23.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-13T19:06:23.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:23.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855364 2026-02-13T19:06:23.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:23.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:06:23.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-13T19:06:23.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:23.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855364 2026-02-13T19:06:23.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-13T19:06:23.954 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855364 2026-02-13T19:06:23.954 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:06:24.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-13T19:06:24.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:24.192 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561028 2026-02-13T19:06:24.193 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:24.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:06:24.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-13T19:06:24.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:24.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561028 2026-02-13T19:06:24.195 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 188978561028 2026-02-13T19:06:24.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-13T19:06:24.195 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:06:24.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-13T19:06:24.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:24.421 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-223338299395 2026-02-13T19:06:24.421 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:24.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:06:24.421 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299395 2026-02-13T19:06:24.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:24.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299395 2026-02-13T19:06:24.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 223338299395' 2026-02-13T19:06:24.423 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 223338299395 2026-02-13T19:06:24.423 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:06:24.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299396 -lt 223338299395 2026-02-13T19:06:24.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:06:24.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:06:24.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:06:24.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:06:24.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:06:24.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:06:24.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:06:24.968 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:06:24.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:06:24.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:06:24.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:06:25.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:06:25.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:06:25.190 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:06:25.190 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:06:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:06:25.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:06:25.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:06:26.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:06:26.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:06:26.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:06:26.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:06:26.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-13T19:06:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-13T19:06:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:06:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:06:26.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=mdata 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:06:26.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:06:26.063 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:06:26.063 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:06:26.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-13T19:06:26.291 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:06:26.291 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=5 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:06:26.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/5/type 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-13T19:06:26.295 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:26.296 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:26.296 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:26.296 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.5.asok 2026-02-13T19:06:26.296 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:06:26.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.5 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:26.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.5.asok 2026-02-13T19:06:26.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:06:26.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:06:26.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:06:26.395 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:06:26.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:06:26.408 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:26.407+0000 7f6148a2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:26.409 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:26.408+0000 7f6148a2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:26.410 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:26.409+0000 7f6148a2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:26.555 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:26.553+0000 7f6148a2f780 -1 Falling back to public interface 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:26.623 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:26.624 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:26.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:26.624 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:26.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:26.883 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:26.882+0000 7f6148a2f780 -1 osd.3 59 log_to_monitors true 2026-02-13T19:06:27.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:27.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:27.846 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:27.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:27.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:27.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [26,59) [v2:127.0.0.1:6826/1413907135,v1:127.0.0.1:6827/1413907135] [v2:127.0.0.1:6828/1413907135,v1:127.0.0.1:6829/1413907135] exists,up 5e295671-cd7d-4e05-a525-61d3fdccbdfc 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:06:28.073 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:06:28.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:06:28.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:06:28.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:06:28.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:06:28.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:06:28.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.383 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:06:28.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-13T19:06:28.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-13T19:06:28.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-13T19:06:28.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.447 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:06:28.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-13T19:06:28.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-13T19:06:28.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-13T19:06:28.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.512 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:06:28.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-13T19:06:28.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-13T19:06:28.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411337' 2026-02-13T19:06:28.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.579 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:06:28.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-13T19:06:28.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-13T19:06:28.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411337 3-270582939651' 2026-02-13T19:06:28.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:06:28.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-13T19:06:28.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-13T19:06:28.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411337 3-270582939651 4-150323855367' 2026-02-13T19:06:28.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.709 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:06:28.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561031 2026-02-13T19:06:28.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561031 2026-02-13T19:06:28.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411337 3-270582939651 4-150323855367 5-188978561031' 2026-02-13T19:06:28.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:06:28.771 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:06:28.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299398 2026-02-13T19:06:28.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299398 2026-02-13T19:06:28.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411337 3-270582939651 4-150323855367 5-188978561031 6-223338299398' 2026-02-13T19:06:28.836 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:28.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-13T19:06:28.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:28.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:06:28.837 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-13T19:06:28.837 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:28.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-13T19:06:28.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-13T19:06:28.839 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803785 2026-02-13T19:06:28.839 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:06:29.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-13T19:06:29.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:06:30.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:06:30.068 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:06:30.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-13T19:06:30.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:06:31.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:06:31.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:06:31.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803785 2026-02-13T19:06:31.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:31.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-13T19:06:31.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:31.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:06:31.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-13T19:06:31.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:31.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-13T19:06:31.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-13T19:06:31.544 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574857 2026-02-13T19:06:31.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:06:31.775 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-13T19:06:31.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:31.776 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411337 2026-02-13T19:06:31.776 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:31.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:06:31.777 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-13T19:06:31.778 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:31.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-13T19:06:31.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-13T19:06:31.779 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411337 2026-02-13T19:06:31.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:06:32.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411337 2026-02-13T19:06:32.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:32.007 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-270582939651 2026-02-13T19:06:32.007 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:32.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:06:32.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-270582939651 2026-02-13T19:06:32.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:32.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=270582939651 2026-02-13T19:06:32.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 270582939651' 2026-02-13T19:06:32.010 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 270582939651 2026-02-13T19:06:32.010 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:06:32.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-13T19:06:32.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:32.234 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855367 2026-02-13T19:06:32.234 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:32.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:06:32.235 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-13T19:06:32.235 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:32.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855367 2026-02-13T19:06:32.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-13T19:06:32.237 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855367 2026-02-13T19:06:32.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:06:32.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855368 -lt 150323855367 2026-02-13T19:06:32.461 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:32.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561031 2026-02-13T19:06:32.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:32.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:06:32.463 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561031 2026-02-13T19:06:32.463 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:32.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561031 2026-02-13T19:06:32.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-13T19:06:32.464 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 188978561031 2026-02-13T19:06:32.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:06:32.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561032 -lt 188978561031 2026-02-13T19:06:32.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:06:32.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-223338299398 2026-02-13T19:06:32.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:06:32.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:06:32.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299398 2026-02-13T19:06:32.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:06:32.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299398 2026-02-13T19:06:32.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 223338299398' 2026-02-13T19:06:32.696 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 223338299398 2026-02-13T19:06:32.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:06:32.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299399 -lt 223338299398 2026-02-13T19:06:32.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:06:32.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:06:32.922 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:06:33.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:06:33.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:06:33.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:06:33.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:06:33.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:06:33.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:06:33.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:06:33.773 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-13T19:06:33.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:06:33.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:06:33.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-13T19:06:33.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:06:33.790 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:06:33.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:06:33.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:06:33.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:06:34.060 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:06:34.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:06:34.348 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:06:34.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:06:34.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:06:34.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:06:34.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:06:34.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:06:34.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:06:34.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:06:34.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:06:34.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:06:34.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:06:34.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:06:34.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:06:34.469 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:06:34.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:06:34.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:06:34.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:06:34.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:34.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:06:34.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:06:34.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:34.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:06:34.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:06:34.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:06:34.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:06:34.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.490 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:06:34.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:06:34.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:06:34.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:06:34.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:06:34.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:06:34.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:06:34.493 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:06:34.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:06:34.494 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:06:34.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:06:34.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:06:34.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:06:34.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:34.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:06:34.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:06:34.497 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:06:34.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:06:34.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:06:34.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:06:34.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:06:34.500 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.500 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:06:34.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:06:34.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:06:34.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:06:34.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:06:34.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:06:34.503 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:06:34.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:06:34.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:06:34.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:06:34.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:06:34.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:34.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:34.522 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:34.523 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.523 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.523 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:34.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:06:34.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:06:34.546 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:06:34.546 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:34.547 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:06:34.548 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:06:34.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:06:34.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:06:34.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:34.591 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:34.592 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.592 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:34.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:06:34.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:06:34.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:06:34.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:06:34.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:34.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:34.732 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:34.732 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:34.733 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:34.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:34.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:06:34.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:06:34.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:06:34.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:06:34.842 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:06:34.849 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:35.852 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:06:35.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:06:35.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:06:35.891 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:06:35.896 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:06:35.897 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:06:34.538+0000 7fb5a02fdc00 0 load: jerasure load: lrc 2026-02-13T19:06:35.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-13T19:06:35.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-13T19:06:35.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-13T19:06:35.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-13T19:06:35.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-13T19:06:35.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:06:35.899 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-13T19:06:35.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:06:35.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:35.905 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:35.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b78a4fb9-c704-4eb4-b94e-ed04d77b6999 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b78a4fb9-c704-4eb4-b94e-ed04d77b6999' 2026-02-13T19:06:35.907 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 b78a4fb9-c704-4eb4-b94e-ed04d77b6999 2026-02-13T19:06:35.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:35.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7do9pjJJTNhAAB+g8QTCx0QZ7ioxqGKgJtw== 2026-02-13T19:06:35.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7do9pjJJTNhAAB+g8QTCx0QZ7ioxqGKgJtw=="}' 2026-02-13T19:06:35.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b78a4fb9-c704-4eb4-b94e-ed04d77b6999 -i td/test-erasure-eio/0/new.json 2026-02-13T19:06:36.013 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:36.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:06:36.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA7do9pjJJTNhAAB+g8QTCx0QZ7ioxqGKgJtw== --osd-uuid b78a4fb9-c704-4eb4-b94e-ed04d77b6999 2026-02-13T19:06:36.037 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.036+0000 7f1d87e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.039 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.038+0000 7f1d87e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.040 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.039+0000 7f1d87e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.040 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.039+0000 7f1d87e2e780 -1 bdev(0x377ec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:36.040 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.039+0000 7f1d87e2e780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:06:36.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:06:36.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:36.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:06:36.492 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:06:36.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:36.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:06:36.594 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:06:36.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:36.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:36.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:36.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:36.609 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.608+0000 7fb7b3835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.611 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.610+0000 7fb7b3835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.611 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.611+0000 7fb7b3835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:36.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:36.692 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:36.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:36.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:06:36.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:36.809 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:36.808+0000 7fb7b3835780 -1 Falling back to public interface 2026-02-13T19:06:37.060 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:37.059+0000 7fb7b3835780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:06:37.790 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:37.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:37.791 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:37.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:37.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:37.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:06:38.014 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4236581261,v1:127.0.0.1:6803/4236581261] [v2:127.0.0.1:6804/4236581261,v1:127.0.0.1:6805/4236581261] exists,up b78a4fb9-c704-4eb4-b94e-ed04d77b6999 2026-02-13T19:06:38.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:38.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:38.020 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:38.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3249481a-4bed-4ae3-9514-63b6f5f062ed 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 3249481a-4bed-4ae3-9514-63b6f5f062ed' 2026-02-13T19:06:38.022 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 3249481a-4bed-4ae3-9514-63b6f5f062ed 2026-02-13T19:06:38.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:38.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+do9pimOgARAAKUjzRsasRlp34XPOXbj6ww== 2026-02-13T19:06:38.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+do9pimOgARAAKUjzRsasRlp34XPOXbj6ww=="}' 2026-02-13T19:06:38.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3249481a-4bed-4ae3-9514-63b6f5f062ed -i td/test-erasure-eio/1/new.json 2026-02-13T19:06:38.259 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:38.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:06:38.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+do9pimOgARAAKUjzRsasRlp34XPOXbj6ww== --osd-uuid 3249481a-4bed-4ae3-9514-63b6f5f062ed 2026-02-13T19:06:38.283 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.282+0000 7f8312209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:38.284 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.283+0000 7f8312209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:38.285 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.284+0000 7f8312209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:38.285 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.284+0000 7f8312209780 -1 bdev(0x2730c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:38.285 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.284+0000 7f8312209780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:06:38.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:06:38.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:38.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:06:38.619 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:06:38.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:38.935 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:06:38.935 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:06:38.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:38.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:38.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:38.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:38.949 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.948+0000 7f0494869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:38.950 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.950+0000 7f0494869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:38.951 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:38.951+0000 7f0494869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:39.093 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:39.092+0000 7f0494869780 -1 Falling back to public interface 2026-02-13T19:06:39.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:39.156 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:39.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:39.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:06:39.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:39.387 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:39.386+0000 7f0494869780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:06:40.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:40.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:40.380 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:40.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:40.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:40.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:06:40.607 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3133364540,v1:127.0.0.1:6811/3133364540] [v2:127.0.0.1:6812/3133364540,v1:127.0.0.1:6813/3133364540] exists,up 3249481a-4bed-4ae3-9514-63b6f5f062ed 2026-02-13T19:06:40.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:40.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:40.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:40.613 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:40.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0dc6df9e-d314-4fb1-b597-0348564a98d7 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0dc6df9e-d314-4fb1-b597-0348564a98d7' 2026-02-13T19:06:40.615 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 0dc6df9e-d314-4fb1-b597-0348564a98d7 2026-02-13T19:06:40.616 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:40.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAdo9pWAjuJBAAKqDgfFNnpQuS3cKRvZH9vg== 2026-02-13T19:06:40.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAdo9pWAjuJBAAKqDgfFNnpQuS3cKRvZH9vg=="}' 2026-02-13T19:06:40.621 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0dc6df9e-d314-4fb1-b597-0348564a98d7 -i td/test-erasure-eio/2/new.json 2026-02-13T19:06:40.850 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:06:40.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:06:40.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAdo9pWAjuJBAAKqDgfFNnpQuS3cKRvZH9vg== --osd-uuid 0dc6df9e-d314-4fb1-b597-0348564a98d7 2026-02-13T19:06:40.874 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:40.874+0000 7f30a942f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:40.876 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:40.875+0000 7f30a942f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:40.877 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:40.876+0000 7f30a942f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:40.877 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:40.876+0000 7f30a942f780 -1 bdev(0x2dfac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:40.877 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:40.876+0000 7f30a942f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:06:41.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:06:41.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:41.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:06:41.195 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:06:41.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:41.515 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:06:41.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:06:41.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:41.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:41.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:41.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:41.531 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:41.530+0000 7efd91445780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:41.532 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:41.531+0000 7efd91445780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:41.533 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:41.532+0000 7efd91445780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:41.653 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:41.652+0000 7efd91445780 -1 Falling back to public interface 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:41.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:41.748 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:41.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:41.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:41.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:06:41.960 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:41.959+0000 7efd91445780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:06:41.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:42.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:42.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:42.973 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:42.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:42.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:42.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:06:43.202 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/375079832,v1:127.0.0.1:6819/375079832] [v2:127.0.0.1:6820/375079832,v1:127.0.0.1:6821/375079832] exists,up 0dc6df9e-d314-4fb1-b597-0348564a98d7 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:43.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:43.207 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:43.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f 2026-02-13T19:06:43.209 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f 2026-02-13T19:06:43.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f' 2026-02-13T19:06:43.210 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:43.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDdo9p3YLPDBAA65BIGI/a8zBjAZ+L4WJZOg== 2026-02-13T19:06:43.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDdo9p3YLPDBAA65BIGI/a8zBjAZ+L4WJZOg=="}' 2026-02-13T19:06:43.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f -i td/test-erasure-eio/3/new.json 2026-02-13T19:06:43.441 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:06:43.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:06:43.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDdo9p3YLPDBAA65BIGI/a8zBjAZ+L4WJZOg== --osd-uuid d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f 2026-02-13T19:06:43.466 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:43.465+0000 7f95c5062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:43.467 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:43.466+0000 7f95c5062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:43.468 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:43.467+0000 7f95c5062780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:43.468 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:43.468+0000 7f95c5062780 -1 bdev(0x40f8c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:43.468 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:43.468+0000 7f95c5062780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:06:43.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:06:43.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:43.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:06:43.798 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:06:43.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:44.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:06:44.116 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:06:44.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:44.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:44.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:44.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:44.130 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:44.129+0000 7fae835fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:44.131 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:44.130+0000 7fae835fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:44.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:44.131+0000 7fae835fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:44.257 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:44.256+0000 7fae835fd780 -1 Falling back to public interface 2026-02-13T19:06:44.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:06:44.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:44.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:06:44.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:44.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:44.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:44.339 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:44.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:44.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:44.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:44.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:44.585 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:44.584+0000 7fae835fd780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:06:45.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:45.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:45.564 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:45.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:45.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:45.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:06:45.789 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/943092638,v1:127.0.0.1:6827/943092638] [v2:127.0.0.1:6828/943092638,v1:127.0.0.1:6829/943092638] exists,up d2c9e8e9-438b-4f0c-b9c0-6de57d64d51f 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:45.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:45.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:45.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:45.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=195effeb-6ab8-4f38-b7b0-ba761f884f06 2026-02-13T19:06:45.797 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 195effeb-6ab8-4f38-b7b0-ba761f884f06 2026-02-13T19:06:45.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 195effeb-6ab8-4f38-b7b0-ba761f884f06' 2026-02-13T19:06:45.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:45.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBFdo9pzbPPLxAAxuyfaBbWdu5UiQgT4PkC1w== 2026-02-13T19:06:45.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBFdo9pzbPPLxAAxuyfaBbWdu5UiQgT4PkC1w=="}' 2026-02-13T19:06:45.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 195effeb-6ab8-4f38-b7b0-ba761f884f06 -i td/test-erasure-eio/4/new.json 2026-02-13T19:06:46.031 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:06:46.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:06:46.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBFdo9pzbPPLxAAxuyfaBbWdu5UiQgT4PkC1w== --osd-uuid 195effeb-6ab8-4f38-b7b0-ba761f884f06 2026-02-13T19:06:46.057 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.056+0000 7ff93341c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.058 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.057+0000 7ff93341c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.059 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.058+0000 7ff93341c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.059 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.058+0000 7ff93341c780 -1 bdev(0x3674c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:46.060 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.059+0000 7ff93341c780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:06:46.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:06:46.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:46.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:06:46.383 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:06:46.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:46.715 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:06:46.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:06:46.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:46.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:46.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:46.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:46.730 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.730+0000 7fa4df1d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.732 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.731+0000 7fa4df1d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.733 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.732+0000 7fa4df1d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:46.862 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:46.860+0000 7fa4df1d6780 -1 Falling back to public interface 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:46.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:46.946 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:46.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:46.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:46.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:06:47.158 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:47.157+0000 7fa4df1d6780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:06:47.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:48.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:48.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:48.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:48.180 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:48.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:48.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:06:48.406 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3927249706,v1:127.0.0.1:6835/3927249706] [v2:127.0.0.1:6836/3927249706,v1:127.0.0.1:6837/3927249706] exists,up 195effeb-6ab8-4f38-b7b0-ba761f884f06 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:48.407 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:48.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-13T19:06:48.409 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:48.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f4764552-7179-42b4-ad6e-082826aaa615 2026-02-13T19:06:48.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f4764552-7179-42b4-ad6e-082826aaa615' 2026-02-13T19:06:48.410 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 f4764552-7179-42b4-ad6e-082826aaa615 2026-02-13T19:06:48.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:48.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIdo9pS/4CGRAAC4Rs4PR9N2lH9JkRs2fttw== 2026-02-13T19:06:48.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIdo9pS/4CGRAAC4Rs4PR9N2lH9JkRs2fttw=="}' 2026-02-13T19:06:48.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f4764552-7179-42b4-ad6e-082826aaa615 -i td/test-erasure-eio/5/new.json 2026-02-13T19:06:48.649 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:06:48.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-13T19:06:48.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIdo9pS/4CGRAAC4Rs4PR9N2lH9JkRs2fttw== --osd-uuid f4764552-7179-42b4-ad6e-082826aaa615 2026-02-13T19:06:48.673 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:48.672+0000 7f119b3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:48.675 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:48.674+0000 7f119b3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:48.675 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:48.675+0000 7f119b3e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:48.676 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:48.675+0000 7f119b3e3780 -1 bdev(0x3424c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:48.676 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:48.675+0000 7f119b3e3780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-13T19:06:49.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-13T19:06:49.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:49.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:06:49.008 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:06:49.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:49.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:06:49.322 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:06:49.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:49.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:49.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:49.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:49.337 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:49.336+0000 7fe873627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:49.338 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:49.338+0000 7fe873627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:49.339 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:49.339+0000 7fe873627780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:49.522 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:49.521+0000 7fe873627780 -1 Falling back to public interface 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:49.555 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:49.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:49.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:49.556 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:06:49.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:49.880 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:49.879+0000 7fe873627780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:06:50.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:50.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:50.784 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:50.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:50.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:50.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:06:51.010 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/898411783,v1:127.0.0.1:6843/898411783] [v2:127.0.0.1:6844/898411783,v1:127.0.0.1:6845/898411783] exists,up f4764552-7179-42b4-ad6e-082826aaa615 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:51.014 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:51.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-13T19:06:51.016 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:51.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c9ba4475-bf66-4916-8663-c17945e93097 2026-02-13T19:06:51.016 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 c9ba4475-bf66-4916-8663-c17945e93097 2026-02-13T19:06:51.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 c9ba4475-bf66-4916-8663-c17945e93097' 2026-02-13T19:06:51.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:51.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLdo9pDnVaARAAopip8V9MoophD08f819mBA== 2026-02-13T19:06:51.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLdo9pDnVaARAAopip8V9MoophD08f819mBA=="}' 2026-02-13T19:06:51.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c9ba4475-bf66-4916-8663-c17945e93097 -i td/test-erasure-eio/6/new.json 2026-02-13T19:06:51.260 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:06:51.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-13T19:06:51.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLdo9pDnVaARAAopip8V9MoophD08f819mBA== --osd-uuid c9ba4475-bf66-4916-8663-c17945e93097 2026-02-13T19:06:51.285 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.284+0000 7f0207e87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:51.286 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.285+0000 7f0207e87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:51.287 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.286+0000 7f0207e87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:51.287 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.286+0000 7f0207e87780 -1 bdev(0x39dec00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:51.287 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.287+0000 7f0207e87780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-13T19:06:51.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-13T19:06:51.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:51.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:06:51.606 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:06:51.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:51.918 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:06:51.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:06:51.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:51.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:51.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:51.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:51.934 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.933+0000 7f5b0882d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:51.935 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.934+0000 7f5b0882d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:51.936 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:51.935+0000 7f5b0882d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:52.123 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:52.121+0000 7f5b0882d780 -1 Falling back to public interface 2026-02-13T19:06:52.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:52.143 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:52.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:52.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:52.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:06:52.372 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:52.461 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:52.460+0000 7f5b0882d780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:06:53.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:53.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:53.373 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:53.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:53.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:53.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:06:53.601 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/4078655754,v1:127.0.0.1:6851/4078655754] [v2:127.0.0.1:6852/4078655754,v1:127.0.0.1:6853/4078655754] exists,up c9ba4475-bf66-4916-8663-c17945e93097 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:53.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:53.606 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:53.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5dfc33b8-027f-421c-95a2-78cf110507ff 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 5dfc33b8-027f-421c-95a2-78cf110507ff' 2026-02-13T19:06:53.608 INFO:tasks.workunit.client.0.trial034.stdout:add osd7 5dfc33b8-027f-421c-95a2-78cf110507ff 2026-02-13T19:06:53.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:53.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBNdo9pXPGdJBAAl9OLBZhtw0+551yU+P0NBA== 2026-02-13T19:06:53.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBNdo9pXPGdJBAAl9OLBZhtw0+551yU+P0NBA=="}' 2026-02-13T19:06:53.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5dfc33b8-027f-421c-95a2-78cf110507ff -i td/test-erasure-eio/7/new.json 2026-02-13T19:06:53.846 INFO:tasks.workunit.client.0.trial034.stdout:7 2026-02-13T19:06:53.854 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-13T19:06:53.855 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBNdo9pXPGdJBAAl9OLBZhtw0+551yU+P0NBA== --osd-uuid 5dfc33b8-027f-421c-95a2-78cf110507ff 2026-02-13T19:06:53.870 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:53.869+0000 7f16a7e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:53.871 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:53.870+0000 7f16a7e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:53.872 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:53.871+0000 7f16a7e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:53.873 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:53.872+0000 7f16a7e0c780 -1 bdev(0x2a16c00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:53.873 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:53.872+0000 7f16a7e0c780 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-13T19:06:54.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-13T19:06:54.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:54.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-13T19:06:54.181 INFO:tasks.workunit.client.0.trial034.stdout:adding osd7 key to auth repository 2026-02-13T19:06:54.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:54.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-13T19:06:54.501 INFO:tasks.workunit.client.0.trial034.stdout:start osd.7 2026-02-13T19:06:54.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:54.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:54.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:54.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:54.516 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:54.515+0000 7f8acec5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:54.517 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:54.517+0000 7f8acec5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:54.518 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:54.517+0000 7f8acec5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:54.658 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:54.657+0000 7f8acec5b780 -1 Falling back to public interface 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:54.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:54.731 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-13T19:06:54.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:55.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:55.022+0000 7f8acec5b780 -1 osd.7 0 log_to_monitors true 2026-02-13T19:06:55.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:55.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:55.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:55.958 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:55.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:55.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-13T19:06:56.182 INFO:tasks.workunit.client.0.trial034.stdout:osd.7 up in weight 1 up_from 60 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3519986497,v1:127.0.0.1:6859/3519986497] [v2:127.0.0.1:6860/3519986497,v1:127.0.0.1:6861/3519986497] exists,up 5dfc33b8-027f-421c-95a2-78cf110507ff 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:06:56.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:56.184 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-13T19:06:56.185 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:06:56.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bb6ac53d-4a48-4ee3-93b4-56b0f5a27965 2026-02-13T19:06:56.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 bb6ac53d-4a48-4ee3-93b4-56b0f5a27965' 2026-02-13T19:06:56.186 INFO:tasks.workunit.client.0.trial034.stdout:add osd8 bb6ac53d-4a48-4ee3-93b4-56b0f5a27965 2026-02-13T19:06:56.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:06:56.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBQdo9pjCeoCxAAOqVUj0BbmOGVFkduHlG+5A== 2026-02-13T19:06:56.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBQdo9pjCeoCxAAOqVUj0BbmOGVFkduHlG+5A=="}' 2026-02-13T19:06:56.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bb6ac53d-4a48-4ee3-93b4-56b0f5a27965 -i td/test-erasure-eio/8/new.json 2026-02-13T19:06:56.431 INFO:tasks.workunit.client.0.trial034.stdout:8 2026-02-13T19:06:56.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-13T19:06:56.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBQdo9pjCeoCxAAOqVUj0BbmOGVFkduHlG+5A== --osd-uuid bb6ac53d-4a48-4ee3-93b4-56b0f5a27965 2026-02-13T19:06:56.454 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:56.454+0000 7fce88625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:56.456 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:56.455+0000 7fce88625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:56.457 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:56.456+0000 7fce88625780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:56.457 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:56.456+0000 7fce88625780 -1 bdev(0x41c8c00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-13T19:06:56.457 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:56.456+0000 7fce88625780 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-13T19:06:56.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-13T19:06:56.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:06:56.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-13T19:06:56.810 INFO:tasks.workunit.client.0.trial034.stdout:adding osd8 key to auth repository 2026-02-13T19:06:56.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:06:57.129 INFO:tasks.workunit.client.0.trial034.stdout:start osd.8 2026-02-13T19:06:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-13T19:06:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:06:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:06:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:06:57.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:06:57.144 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:57.143+0000 7fb7375fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:57.146 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:57.145+0000 7fb7375fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:57.147 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:57.146+0000 7fb7375fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:06:57.295 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:57.294+0000 7fb7375fb780 -1 Falling back to public interface 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:06:57.356 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:06:57.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:57.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:06:57.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:57.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-13T19:06:57.575 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:06:57.574+0000 7fb7375fb780 -1 osd.8 0 log_to_monitors true 2026-02-13T19:06:57.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:06:58.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:06:58.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:06:58.582 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:06:58.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:06:58.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:06:58.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-13T19:06:58.807 INFO:tasks.workunit.client.0.trial034.stdout:osd.8 up in weight 1 up_from 71 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/3823677216,v1:127.0.0.1:6867/3823677216] [v2:127.0.0.1:6868/3823677216,v1:127.0.0.1:6869/3823677216] exists,up bb6ac53d-4a48-4ee3-93b4-56b0f5a27965 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:06:58.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:06:58.810 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:06:58.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:06:58.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:06:58.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:06:58.854 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:06:58.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:06:58.862 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:06:36.811+0000 7fb7b3835780 0 load: jerasure load: lrc 2026-02-13T19:06:58.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:06:58.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-13T19:06:59.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:06:59.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:06:59.548 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:06:59.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:07:00.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:07:00.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:07:00.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:07:00.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:07:00.560 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:07:00.561 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:07:00.561 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:07:00.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:07:00.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:07:00.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:07:00.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:07:00.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:07:00.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:07:00.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:07:00.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:07:00.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:07:00.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:8' 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:00.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:07:00.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-13T19:07:00.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-13T19:07:00.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783' 2026-02-13T19:07:00.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:00.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:07:00.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-13T19:07:00.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-13T19:07:00.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855' 2026-02-13T19:07:00.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:00.999 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:07:01.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378630 2026-02-13T19:07:01.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378630 2026-02-13T19:07:01.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630' 2026-02-13T19:07:01.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:07:01.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-13T19:07:01.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-13T19:07:01.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702' 2026-02-13T19:07:01.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:07:01.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855365 2026-02-13T19:07:01.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855365 2026-02-13T19:07:01.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-150323855365' 2026-02-13T19:07:01.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.193 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:07:01.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-13T19:07:01.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-13T19:07:01.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-150323855365 5-184683593733' 2026-02-13T19:07:01.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:07:01.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332101 2026-02-13T19:07:01.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332101 2026-02-13T19:07:01.322 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-150323855365 5-184683593733 6-219043332101' 2026-02-13T19:07:01.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.323 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:07:01.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037764 2026-02-13T19:07:01.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037764 2026-02-13T19:07:01.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-150323855365 5-184683593733 6-219043332101 7-257698037764' 2026-02-13T19:07:01.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:01.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:07:01.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678019 2026-02-13T19:07:01.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678019 2026-02-13T19:07:01.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-81604378630 3-111669149702 4-150323855365 5-184683593733 6-219043332101 7-257698037764 8-304942678019' 2026-02-13T19:07:01.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:01.452 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803783 2026-02-13T19:07:01.452 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:01.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:07:01.453 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-13T19:07:01.453 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:01.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803783 2026-02-13T19:07:01.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-13T19:07:01.454 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803783 2026-02-13T19:07:01.455 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:01.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-13T19:07:01.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:07:02.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:07:02.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:02.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-13T19:07:02.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:02.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-13T19:07:02.905 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:02.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:07:02.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-13T19:07:02.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:02.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-13T19:07:02.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-13T19:07:02.908 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574855 2026-02-13T19:07:02.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:07:03.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-13T19:07:03.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:03.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378630 2026-02-13T19:07:03.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:03.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:07:03.131 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378630 2026-02-13T19:07:03.131 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:03.132 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378630 2026-02-13T19:07:03.133 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378630 2026-02-13T19:07:03.133 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378630' 2026-02-13T19:07:03.133 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:07:03.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378630 -lt 81604378630 2026-02-13T19:07:03.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:03.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-13T19:07:03.355 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:03.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:07:03.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-13T19:07:03.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:03.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-13T19:07:03.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-13T19:07:03.358 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149702 2026-02-13T19:07:03.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:07:03.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-13T19:07:03.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:03.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855365 2026-02-13T19:07:03.585 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:03.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:07:03.586 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855365 2026-02-13T19:07:03.586 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:03.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855365 2026-02-13T19:07:03.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855365' 2026-02-13T19:07:03.587 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855365 2026-02-13T19:07:03.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:07:03.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-13T19:07:03.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:03.812 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593733 2026-02-13T19:07:03.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:03.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:07:03.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593733 2026-02-13T19:07:03.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:03.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593733 2026-02-13T19:07:03.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593733' 2026-02-13T19:07:03.815 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593733 2026-02-13T19:07:03.816 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:07:04.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-13T19:07:04.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:04.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332101 2026-02-13T19:07:04.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:04.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:07:04.045 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332101 2026-02-13T19:07:04.045 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:04.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332101 2026-02-13T19:07:04.046 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 219043332101 2026-02-13T19:07:04.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332101' 2026-02-13T19:07:04.046 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:07:04.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332101 -lt 219043332101 2026-02-13T19:07:04.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:04.270 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-257698037764 2026-02-13T19:07:04.270 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:04.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-13T19:07:04.272 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-257698037764 2026-02-13T19:07:04.272 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:04.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037764 2026-02-13T19:07:04.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 257698037764' 2026-02-13T19:07:04.273 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.7 seq 257698037764 2026-02-13T19:07:04.274 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-13T19:07:04.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037764 -lt 257698037764 2026-02-13T19:07:04.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:04.498 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-304942678019 2026-02-13T19:07:04.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:04.499 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:07:04.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-304942678019 2026-02-13T19:07:04.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=304942678019 2026-02-13T19:07:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 304942678019' 2026-02-13T19:07:04.501 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 304942678019 2026-02-13T19:07:04.501 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:07:04.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 304942678019 -lt 304942678019 2026-02-13T19:07:04.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:07:04.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:04.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:05.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:07:05.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:07:05.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:07:05.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:07:05.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:07:05.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:07:05.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:07:05.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:07:05.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:07:05.261 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:07:05.261 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:05.262 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:07:05.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-13T19:07:05.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:07:05.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:07:05.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-13T19:07:05.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-13T19:07:05.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:07:05.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:07:05.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:07:05.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:07:05.823 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:07:05.824 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:05.824 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:05.824 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:05.824 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:07:05.824 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:07:05.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:07:05.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:07:05.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:07:05.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:07:05.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:07:05.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:07:05.872 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:05.873 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:05.873 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:05.873 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:07:05.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:07:05.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:07:05.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=3 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:07:05.921 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:07:06.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-13T19:07:06.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:07:06.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=0 2026-02-13T19:07:06.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:07:06.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/0/type 2026-02-13T19:07:06.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:06.151 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:06.152 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:06.152 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:07:06.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:07:06.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:07:06.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:07:06.197 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:07:06.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.0 2026-02-13T19:07:06.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:07:06.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:07:06.198 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:06.198 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:06.198 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:06.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:07:06.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:07:06.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-13T19:07:06.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:07:06.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=4 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:07:06.249 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:06.476 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=6 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/6/type 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-13T19:07:06.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:07:06.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:07:06.478 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:07:06.478 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-13T19:07:06.478 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-13T19:07:06.478 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-13T19:07:06.479 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:06.479 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:06.479 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:06.479 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.6.asok 2026-02-13T19:07:06.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:07:06.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:07:06.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:07:06.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:07:06.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.6 2026-02-13T19:07:06.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-13T19:07:06.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-13T19:07:06.527 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:06.528 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:06.528 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:06.528 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.6.asok 2026-02-13T19:07:06.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:07:06.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-13T19:07:06.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-13T19:07:06.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:07:06.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:07:06.576 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:07:06.576 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:06.800 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:7' 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:07:06.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:07:07.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:07:07.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-13T19:07:07.329 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already down. 2026-02-13T19:07:07.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-13T19:07:07.585 INFO:tasks.workunit.client.0.trial034.stderr:osd.7 is already out. 2026-02-13T19:07:07.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-13T19:07:07.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:07:07.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:07:07.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:07:07.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:07:07.594 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:07:07.594 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:07:07.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:07:07.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:07:07.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:07:07.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:07:07.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:07:07.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:07:07.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:07:07.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:07:07.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:07:07.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:6 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:7 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:8' 2026-02-13T19:07:07.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:07:07.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:07.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:07:07.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-13T19:07:07.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-13T19:07:07.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-13T19:07:07.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:07.971 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:07:08.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-13T19:07:08.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-13T19:07:08.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-13T19:07:08.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:07:08.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378633 2026-02-13T19:07:08.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378633 2026-02-13T19:07:08.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633' 2026-02-13T19:07:08.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:07:08.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-13T19:07:08.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-13T19:07:08.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705' 2026-02-13T19:07:08.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.163 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:07:08.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-13T19:07:08.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-13T19:07:08.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-150323855368' 2026-02-13T19:07:08.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.228 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:07:08.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593736 2026-02-13T19:07:08.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593736 2026-02-13T19:07:08.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-150323855368 5-184683593736' 2026-02-13T19:07:08.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:07:08.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332104 2026-02-13T19:07:08.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332104 2026-02-13T19:07:08.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-150323855368 5-184683593736 6-219043332104' 2026-02-13T19:07:08.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.358 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:08.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-13T19:07:08.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678022 2026-02-13T19:07:08.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678022 2026-02-13T19:07:08.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-81604378633 3-111669149705 4-150323855368 5-184683593736 6-219043332104 8-304942678022' 2026-02-13T19:07:08.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:08.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-13T19:07:08.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:08.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:07:08.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-13T19:07:08.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:08.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-13T19:07:08.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-13T19:07:08.473 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803786 2026-02-13T19:07:08.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:08.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-13T19:07:08.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:07:09.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:07:09.705 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:09.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803786 2026-02-13T19:07:09.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:09.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574858 2026-02-13T19:07:09.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:09.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:07:09.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-13T19:07:09.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:09.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574858 2026-02-13T19:07:09.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-13T19:07:09.932 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574858 2026-02-13T19:07:09.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:07:10.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-13T19:07:10.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:10.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378633 2026-02-13T19:07:10.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:10.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:07:10.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378633 2026-02-13T19:07:10.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:10.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378633 2026-02-13T19:07:10.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378633' 2026-02-13T19:07:10.160 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378633 2026-02-13T19:07:10.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:07:10.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378634 -lt 81604378633 2026-02-13T19:07:10.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:10.389 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149705 2026-02-13T19:07:10.389 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:10.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:07:10.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-13T19:07:10.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:10.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149705 2026-02-13T19:07:10.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-13T19:07:10.392 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149705 2026-02-13T19:07:10.392 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:07:10.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-13T19:07:10.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:10.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855368 2026-02-13T19:07:10.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:10.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:07:10.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855368 2026-02-13T19:07:10.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:10.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855368 2026-02-13T19:07:10.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-13T19:07:10.621 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855368 2026-02-13T19:07:10.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:07:10.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855369 -lt 150323855368 2026-02-13T19:07:10.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:10.848 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593736 2026-02-13T19:07:10.848 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:10.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:07:10.849 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593736 2026-02-13T19:07:10.849 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:10.850 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593736 2026-02-13T19:07:10.850 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593736' 2026-02-13T19:07:10.851 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593736 2026-02-13T19:07:10.851 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:07:11.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593736 -lt 184683593736 2026-02-13T19:07:11.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:11.080 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332104 2026-02-13T19:07:11.080 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:11.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:07:11.081 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332104 2026-02-13T19:07:11.081 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:11.082 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332104 2026-02-13T19:07:11.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332104' 2026-02-13T19:07:11.083 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 219043332104 2026-02-13T19:07:11.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:07:11.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332105 -lt 219043332104 2026-02-13T19:07:11.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:11.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-304942678022 2026-02-13T19:07:11.309 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:11.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-13T19:07:11.310 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-304942678022 2026-02-13T19:07:11.311 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:11.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=304942678022 2026-02-13T19:07:11.312 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.8 seq 304942678022 2026-02-13T19:07:11.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 304942678022' 2026-02-13T19:07:11.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-13T19:07:11.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 304942678023 -lt 304942678022 2026-02-13T19:07:11.540 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:07:11.540 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:11.540 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:11.854 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:07:11.854 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:07:11.854 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:07:11.855 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:07:11.855 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:07:11.855 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:07:11.855 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:07:11.855 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:07:12.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:07:12.077 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:07:12.077 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:12.077 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-13T19:07:12.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:07:12.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:07:12.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-13T19:07:12.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:07:12.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:07:12.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:07:12.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:07:12.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:07:12.691 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:07:12.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:07:13.002 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:07:13.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:07:13.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:07:13.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:07:13.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:07:13.124 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:07:13.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:07:13.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:07:13.125 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:07:13.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:13.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:07:13.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:07:13.127 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:13.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:07:13.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:07:13.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:07:13.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:07:13.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:07:13.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:07:13.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:07:13.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:07:13.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:07:13.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:07:13.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:07:13.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:07:13.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:07:13.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:07:13.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:07:13.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:13.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:07:13.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:07:13.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:13.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:07:13.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:07:13.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:07:13.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:07:13.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:07:13.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:07:13.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:07:13.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:07:13.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:07:13.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:07:13.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:07:13.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.181 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.182 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:13.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:07:13.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:07:13.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:07:13.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:07:13.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:07:13.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:07:13.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:07:13.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:07:13.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:07:13.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:07:13.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:07:13.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:07:13.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:13.253 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:07:13.253 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:07:13.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:07:13.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:07:13.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:07:13.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:07:13.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:07:13.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:07:13.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:07:13.396 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:13.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:13.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:07:13.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:07:13.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:07:13.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:07:13.505 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:07:13.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:07:14.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:07:14.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:14.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:07:14.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:07:14.553 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:07:14.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:07:14.558 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:07:13.196+0000 7f416b860c00 0 load: jerasure load: lrc 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:07:14.560 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-13T19:07:14.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-13T19:07:14.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:14.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:07:14.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:14.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:14.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e24085b2-606d-40e9-85c9-8854718eeeea 2026-02-13T19:07:14.568 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 e24085b2-606d-40e9-85c9-8854718eeeea 2026-02-13T19:07:14.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e24085b2-606d-40e9-85c9-8854718eeeea' 2026-02-13T19:07:14.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:14.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBido9pIC5PIhAAkQloFz7vObTwGNapnqpawg== 2026-02-13T19:07:14.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBido9pIC5PIhAAkQloFz7vObTwGNapnqpawg=="}' 2026-02-13T19:07:14.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e24085b2-606d-40e9-85c9-8854718eeeea -i td/test-erasure-eio/0/new.json 2026-02-13T19:07:14.681 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:14.687 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:07:14.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBido9pIC5PIhAAkQloFz7vObTwGNapnqpawg== --osd-uuid e24085b2-606d-40e9-85c9-8854718eeeea 2026-02-13T19:07:14.706 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:14.704+0000 7f15b1c73780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:14.707 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:14.705+0000 7f15b1c73780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:14.708 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:14.706+0000 7f15b1c73780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:14.708 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:14.706+0000 7f15b1c73780 -1 bdev(0x2532c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:14.709 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:14.706+0000 7f15b1c73780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:07:15.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:07:15.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:15.121 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:07:15.121 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:07:15.121 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:15.225 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:07:15.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:07:15.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:15.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:15.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:15.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:15.244 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:15.242+0000 7feb49e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:15.245 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:15.243+0000 7feb49e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:15.246 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:15.244+0000 7feb49e69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:15.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:15.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:15.327 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:15.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:15.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:07:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:15.448 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:15.446+0000 7feb49e69780 -1 Falling back to public interface 2026-02-13T19:07:15.693 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:15.691+0000 7feb49e69780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:07:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:16.431 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:07:16.662 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3785911135,v1:127.0.0.1:6803/3785911135] [v2:127.0.0.1:6804/3785911135,v1:127.0.0.1:6805/3785911135] exists,up e24085b2-606d-40e9-85c9-8854718eeeea 2026-02-13T19:07:16.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:16.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:16.668 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:16.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1a9efa7c-ebde-457a-b5e9-4ce9857872b4 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1a9efa7c-ebde-457a-b5e9-4ce9857872b4' 2026-02-13T19:07:16.670 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 1a9efa7c-ebde-457a-b5e9-4ce9857872b4 2026-02-13T19:07:16.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:16.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBkdo9p2l5VKBAAJX5SfNcv+oYeG2JljNwOKw== 2026-02-13T19:07:16.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBkdo9p2l5VKBAAJX5SfNcv+oYeG2JljNwOKw=="}' 2026-02-13T19:07:16.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1a9efa7c-ebde-457a-b5e9-4ce9857872b4 -i td/test-erasure-eio/1/new.json 2026-02-13T19:07:16.910 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:16.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:07:16.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBkdo9p2l5VKBAAJX5SfNcv+oYeG2JljNwOKw== --osd-uuid 1a9efa7c-ebde-457a-b5e9-4ce9857872b4 2026-02-13T19:07:16.937 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:16.936+0000 7f3d13862780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:16.938 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:16.938+0000 7f3d13862780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:16.939 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:16.938+0000 7f3d13862780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:16.940 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:16.939+0000 7f3d13862780 -1 bdev(0x3316c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:16.940 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:16.939+0000 7f3d13862780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:07:17.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:07:17.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:17.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:07:17.255 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:07:17.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:17.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:07:17.584 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:07:17.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:17.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:17.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:17.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:17.601 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:17.600+0000 7f3357e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:17.602 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:17.601+0000 7f3357e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:17.603 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:17.602+0000 7f3357e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:17.741 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:17.740+0000 7f3357e0f780 -1 Falling back to public interface 2026-02-13T19:07:17.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:07:17.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:17.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:07:17.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:17.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:17.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:17.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:17.809 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:17.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:17.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:07:18.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:18.130 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:18.129+0000 7f3357e0f780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:07:19.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:19.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:19.036 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:19.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:19.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:19.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:07:19.268 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2032317137,v1:127.0.0.1:6811/2032317137] [v2:127.0.0.1:6812/2032317137,v1:127.0.0.1:6813/2032317137] exists,up 1a9efa7c-ebde-457a-b5e9-4ce9857872b4 2026-02-13T19:07:19.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:19.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:19.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:19.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:07:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:07:19.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:19.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:19.279 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:19.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:19.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:07:19.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:19.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8c2a9143-aa91-4207-8519-70d1347ca186 2026-02-13T19:07:19.283 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 8c2a9143-aa91-4207-8519-70d1347ca186 2026-02-13T19:07:19.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8c2a9143-aa91-4207-8519-70d1347ca186' 2026-02-13T19:07:19.285 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:19.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBndo9pjALiEBAAf6MD9jLJoWVueZRrK9qoyg== 2026-02-13T19:07:19.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBndo9pjALiEBAAf6MD9jLJoWVueZRrK9qoyg=="}' 2026-02-13T19:07:19.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8c2a9143-aa91-4207-8519-70d1347ca186 -i td/test-erasure-eio/2/new.json 2026-02-13T19:07:19.516 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:07:19.524 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:07:19.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBndo9pjALiEBAAf6MD9jLJoWVueZRrK9qoyg== --osd-uuid 8c2a9143-aa91-4207-8519-70d1347ca186 2026-02-13T19:07:19.544 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:19.543+0000 7f3b3302b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:19.546 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:19.545+0000 7f3b3302b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:19.547 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:19.546+0000 7f3b3302b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:19.547 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:19.546+0000 7f3b3302b780 -1 bdev(0x2a84c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:19.547 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:19.546+0000 7f3b3302b780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:07:19.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:07:19.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:19.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:07:19.866 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:07:19.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:20.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:07:20.187 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:07:20.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:20.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:20.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:20.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:20.205 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:20.204+0000 7ff83782d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:20.206 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:20.206+0000 7ff83782d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:20.207 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:20.206+0000 7ff83782d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:20.336 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:20.335+0000 7ff83782d780 -1 Falling back to public interface 2026-02-13T19:07:20.419 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:07:20.419 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:20.420 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:20.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:20.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:07:20.625 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:20.624+0000 7ff83782d780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:07:20.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:21.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:21.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:21.652 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:21.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:21.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:21.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:07:21.879 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3484058117,v1:127.0.0.1:6819/3484058117] [v2:127.0.0.1:6820/3484058117,v1:127.0.0.1:6821/3484058117] exists,up 8c2a9143-aa91-4207-8519-70d1347ca186 2026-02-13T19:07:21.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:21.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:21.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:21.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:21.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:07:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:21.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:21.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:07:21.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:21.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8497120-aed3-4a06-b179-43efb2b909d8 2026-02-13T19:07:21.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 a8497120-aed3-4a06-b179-43efb2b909d8' 2026-02-13T19:07:21.893 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 a8497120-aed3-4a06-b179-43efb2b909d8 2026-02-13T19:07:21.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:21.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBpdo9pjAY7NRAAAdXAVD/s4C0VwsdZyG0CzA== 2026-02-13T19:07:21.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBpdo9pjAY7NRAAAdXAVD/s4C0VwsdZyG0CzA=="}' 2026-02-13T19:07:21.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8497120-aed3-4a06-b179-43efb2b909d8 -i td/test-erasure-eio/3/new.json 2026-02-13T19:07:22.126 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:07:22.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:07:22.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBpdo9pjAY7NRAAAdXAVD/s4C0VwsdZyG0CzA== --osd-uuid a8497120-aed3-4a06-b179-43efb2b909d8 2026-02-13T19:07:22.152 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.151+0000 7fc72e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.153 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.153+0000 7fc72e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.154 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.154+0000 7fc72e1d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.154 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.154+0000 7fc72e1d8780 -1 bdev(0x34b6c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:22.155 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.154+0000 7fc72e1d8780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:07:22.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:07:22.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:22.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:07:22.475 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:07:22.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:22.811 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:07:22.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:07:22.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:22.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:22.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:22.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:22.829 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.828+0000 7f0419a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.830 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.829+0000 7f0419a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.831 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.830+0000 7f0419a33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:22.971 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:22.970+0000 7f0419a33780 -1 Falling back to public interface 2026-02-13T19:07:23.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:07:23.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:23.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:07:23.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:23.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:23.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:23.043 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:23.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:23.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:23.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:07:23.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:23.297 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:23.296+0000 7f0419a33780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:07:24.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:24.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:24.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:24.270 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:24.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:24.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:07:24.504 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2760230009,v1:127.0.0.1:6827/2760230009] [v2:127.0.0.1:6828/2760230009,v1:127.0.0.1:6829/2760230009] exists,up a8497120-aed3-4a06-b179-43efb2b909d8 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:07:24.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:24.509 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:24.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:24.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cd8c21a7-0e00-4869-a520-d00ef4764691 2026-02-13T19:07:24.512 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 cd8c21a7-0e00-4869-a520-d00ef4764691 2026-02-13T19:07:24.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 cd8c21a7-0e00-4869-a520-d00ef4764691' 2026-02-13T19:07:24.512 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:24.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBsdo9p5hLlHhAANa1x9d5ReHtBKx8UlPW7pQ== 2026-02-13T19:07:24.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBsdo9p5hLlHhAANa1x9d5ReHtBKx8UlPW7pQ=="}' 2026-02-13T19:07:24.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cd8c21a7-0e00-4869-a520-d00ef4764691 -i td/test-erasure-eio/4/new.json 2026-02-13T19:07:24.748 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:07:24.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:07:24.757 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBsdo9p5hLlHhAANa1x9d5ReHtBKx8UlPW7pQ== --osd-uuid cd8c21a7-0e00-4869-a520-d00ef4764691 2026-02-13T19:07:24.774 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:24.773+0000 7f1f40459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:24.775 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:24.774+0000 7f1f40459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:24.776 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:24.775+0000 7f1f40459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:24.777 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:24.775+0000 7f1f40459780 -1 bdev(0x3ba8c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:24.777 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:24.776+0000 7f1f40459780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:07:25.087 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:07:25.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:25.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:07:25.089 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:07:25.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:25.405 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:07:25.405 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:07:25.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:25.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:25.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:25.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:25.424 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:25.423+0000 7fc17400a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:25.425 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:25.424+0000 7fc17400a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:25.426 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:25.425+0000 7fc17400a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:25.554 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:25.552+0000 7fc17400a780 -1 Falling back to public interface 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:25.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:25.635 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:25.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:25.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:07:25.828 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:25.827+0000 7fc17400a780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:07:25.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:26.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:07:27.090 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/582007350,v1:127.0.0.1:6835/582007350] [v2:127.0.0.1:6836/582007350,v1:127.0.0.1:6837/582007350] exists,up cd8c21a7-0e00-4869-a520-d00ef4764691 2026-02-13T19:07:27.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:27.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:27.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:27.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:27.096 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:27.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=229aad4f-e1a2-440d-ac5c-1efd38424f42 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 229aad4f-e1a2-440d-ac5c-1efd38424f42' 2026-02-13T19:07:27.098 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 229aad4f-e1a2-440d-ac5c-1efd38424f42 2026-02-13T19:07:27.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:27.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBvdo9pl0U+BhAAqbTDzakM+5po6+VZMhJXsg== 2026-02-13T19:07:27.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBvdo9pl0U+BhAAqbTDzakM+5po6+VZMhJXsg=="}' 2026-02-13T19:07:27.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 229aad4f-e1a2-440d-ac5c-1efd38424f42 -i td/test-erasure-eio/5/new.json 2026-02-13T19:07:27.343 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:07:27.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-13T19:07:27.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvdo9pl0U+BhAAqbTDzakM+5po6+VZMhJXsg== --osd-uuid 229aad4f-e1a2-440d-ac5c-1efd38424f42 2026-02-13T19:07:27.369 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:27.368+0000 7f238f846780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:27.370 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:27.370+0000 7f238f846780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:27.371 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:27.370+0000 7f238f846780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:27.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:27.371+0000 7f238f846780 -1 bdev(0x2410c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:27.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:27.371+0000 7f238f846780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-13T19:07:27.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-13T19:07:27.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:27.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:07:27.682 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:07:27.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:27.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:07:28.000 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:07:28.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:28.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:28.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:28.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:28.016 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:28.015+0000 7fd4565d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:28.017 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:28.016+0000 7fd4565d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:28.018 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:28.017+0000 7fd4565d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:28.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:07:28.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:28.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:07:28.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:28.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:28.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:28.229 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:28.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:28.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:28.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:07:28.236 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:28.236+0000 7fd4565d8780 -1 Falling back to public interface 2026-02-13T19:07:28.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:28.591 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:28.590+0000 7fd4565d8780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:07:29.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:29.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:29.465 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:29.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:29.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:29.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:07:29.691 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/192565349,v1:127.0.0.1:6843/192565349] [v2:127.0.0.1:6844/192565349,v1:127.0.0.1:6845/192565349] exists,up 229aad4f-e1a2-440d-ac5c-1efd38424f42 2026-02-13T19:07:29.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:29.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:29.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:29.697 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:29.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b8eb62a-2f6d-4fec-808a-0a0ef85b606f 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 5b8eb62a-2f6d-4fec-808a-0a0ef85b606f' 2026-02-13T19:07:29.699 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 5b8eb62a-2f6d-4fec-808a-0a0ef85b606f 2026-02-13T19:07:29.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:29.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBxdo9p3P8WKhAAvVdjMaqj3V0TL7Z0P4vKfA== 2026-02-13T19:07:29.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBxdo9p3P8WKhAAvVdjMaqj3V0TL7Z0P4vKfA=="}' 2026-02-13T19:07:29.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b8eb62a-2f6d-4fec-808a-0a0ef85b606f -i td/test-erasure-eio/6/new.json 2026-02-13T19:07:29.936 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:07:29.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-13T19:07:29.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBxdo9p3P8WKhAAvVdjMaqj3V0TL7Z0P4vKfA== --osd-uuid 5b8eb62a-2f6d-4fec-808a-0a0ef85b606f 2026-02-13T19:07:29.963 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:29.962+0000 7f70fc3d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:29.964 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:29.964+0000 7f70fc3d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:29.965 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:29.964+0000 7f70fc3d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:29.966 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:29.965+0000 7f70fc3d8780 -1 bdev(0x2e64c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:29.966 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:29.965+0000 7f70fc3d8780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-13T19:07:30.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-13T19:07:30.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:30.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:07:30.286 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:07:30.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:30.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:07:30.608 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:07:30.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:30.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:30.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:30.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:30.626 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:30.625+0000 7fb30cc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:30.627 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:30.626+0000 7fb30cc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:30.628 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:30.627+0000 7fb30cc48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:30.751 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:30.750+0000 7fb30cc48780 -1 Falling back to public interface 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:30.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:30.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:30.841 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:30.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:30.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:07:31.052 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:31.051+0000 7fb30cc48780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:07:31.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:32.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:32.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:32.069 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:32.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:32.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:32.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:07:32.300 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/150990039,v1:127.0.0.1:6851/150990039] [v2:127.0.0.1:6852/150990039,v1:127.0.0.1:6853/150990039] exists,up 5b8eb62a-2f6d-4fec-808a-0a0ef85b606f 2026-02-13T19:07:32.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:32.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:32.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:32.303 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:32.304 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:07:32.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:07:32.305 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:07:32.345 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:07:32.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:07:32.352 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:07:15.449+0000 7feb49e69780 0 load: jerasure load: lrc 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:07:32.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:07:32.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:07:32.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:07:33.011 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:07:33.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:07:34.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:07:34.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:07:34.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:07:34.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:07:34.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:07:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:07:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:07:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:07:34.107 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:07:34.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:07:34.331 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:34.331 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:34.331 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:07:34.331 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:34.332 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:34.332 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:07:34.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:07:34.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.332 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:07:34.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-13T19:07:34.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-13T19:07:34.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-13T19:07:34.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.396 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:07:34.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-13T19:07:34.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-13T19:07:34.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855' 2026-02-13T19:07:34.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:07:34.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-13T19:07:34.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-13T19:07:34.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-81604378629' 2026-02-13T19:07:34.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.523 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:07:34.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-13T19:07:34.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-13T19:07:34.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-81604378629 3-111669149701' 2026-02-13T19:07:34.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.587 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:07:34.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-13T19:07:34.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-13T19:07:34.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-81604378629 3-111669149701 4-146028888068' 2026-02-13T19:07:34.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.651 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:07:34.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-13T19:07:34.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-13T19:07:34.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-81604378629 3-111669149701 4-146028888068 5-180388626435' 2026-02-13T19:07:34.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:34.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:07:34.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-13T19:07:34.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-13T19:07:34.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-81604378629 3-111669149701 4-146028888068 5-180388626435 6-214748364803' 2026-02-13T19:07:34.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:34.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-13T19:07:34.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:34.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:07:34.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-13T19:07:34.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:34.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-13T19:07:34.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-13T19:07:34.786 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803782 2026-02-13T19:07:34.786 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:35.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-13T19:07:35.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:07:36.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:07:36.012 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-13T19:07:36.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:36.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-13T19:07:36.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:36.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:07:36.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-13T19:07:36.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:36.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-13T19:07:36.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-13T19:07:36.241 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574855 2026-02-13T19:07:36.242 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:07:36.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-13T19:07:36.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:36.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378629 2026-02-13T19:07:36.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:36.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:07:36.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-13T19:07:36.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:36.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378629 2026-02-13T19:07:36.472 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378629 2026-02-13T19:07:36.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-13T19:07:36.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:07:36.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-13T19:07:36.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:36.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-13T19:07:36.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:36.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:07:36.701 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-13T19:07:36.701 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:36.702 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-13T19:07:36.702 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-13T19:07:36.702 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149701 2026-02-13T19:07:36.703 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:07:36.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-13T19:07:36.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:36.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888068 2026-02-13T19:07:36.927 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:36.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:07:36.928 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-13T19:07:36.928 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:36.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888068 2026-02-13T19:07:36.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-13T19:07:36.929 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888068 2026-02-13T19:07:36.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:07:37.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-13T19:07:37.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:37.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626435 2026-02-13T19:07:37.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:37.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:07:37.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626435 2026-02-13T19:07:37.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:37.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626435 2026-02-13T19:07:37.155 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 180388626435 2026-02-13T19:07:37.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626435' 2026-02-13T19:07:37.156 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:07:37.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626436 -lt 180388626435 2026-02-13T19:07:37.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:37.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364803 2026-02-13T19:07:37.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:37.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:07:37.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364803 2026-02-13T19:07:37.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:37.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-13T19:07:37.386 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 214748364803 2026-02-13T19:07:37.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-13T19:07:37.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:07:37.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-13T19:07:37.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:07:37.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:37.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:37.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:07:37.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:07:38.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:07:38.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:07:38.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:38.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:38.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:07:38.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:07:38.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:07:38.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:07:38.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:07:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:07:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:07:38.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:07:38.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-13T19:07:38.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-13T19:07:38.797 INFO:tasks.workunit.client.0.trial034.stderr:osd.0 is already out. 2026-02-13T19:07:38.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-13T19:07:38.805 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:07:38.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:07:38.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:07:38.806 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:07:38.806 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:07:38.806 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:07:38.807 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:07:38.807 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:07:38.807 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:07:38.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:07:39.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-13T19:07:39.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-13T19:07:39.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-13T19:07:39.185 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.185 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:07:39.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-13T19:07:39.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-13T19:07:39.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-13T19:07:39.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.250 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:07:39.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378632 2026-02-13T19:07:39.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378632 2026-02-13T19:07:39.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632' 2026-02-13T19:07:39.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.316 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:07:39.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149703 2026-02-13T19:07:39.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149703 2026-02-13T19:07:39.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149703' 2026-02-13T19:07:39.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.378 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:07:39.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-13T19:07:39.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-13T19:07:39.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149703 4-146028888071' 2026-02-13T19:07:39.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:07:39.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626438 2026-02-13T19:07:39.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626438 2026-02-13T19:07:39.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149703 4-146028888071 5-180388626438' 2026-02-13T19:07:39.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:07:39.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:07:39.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364806 2026-02-13T19:07:39.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364806 2026-02-13T19:07:39.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149703 4-146028888071 5-180388626438 6-214748364806' 2026-02-13T19:07:39.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:39.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-13T19:07:39.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:39.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:07:39.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-13T19:07:39.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:39.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-13T19:07:39.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-13T19:07:39.570 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803785 2026-02-13T19:07:39.571 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:07:39.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-13T19:07:39.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:39.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-13T19:07:39.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:39.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:07:39.796 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-13T19:07:39.796 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:39.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-13T19:07:39.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-13T19:07:39.798 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574857 2026-02-13T19:07:39.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:07:40.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-13T19:07:40.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:40.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378632 2026-02-13T19:07:40.024 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:40.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:07:40.025 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378632 2026-02-13T19:07:40.025 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:40.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378632 2026-02-13T19:07:40.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378632' 2026-02-13T19:07:40.027 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378632 2026-02-13T19:07:40.027 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:07:40.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-13T19:07:40.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:40.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149703 2026-02-13T19:07:40.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:40.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:07:40.253 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149703 2026-02-13T19:07:40.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:40.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149703 2026-02-13T19:07:40.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-13T19:07:40.255 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149703 2026-02-13T19:07:40.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:07:40.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-13T19:07:40.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:40.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888071 2026-02-13T19:07:40.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:40.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:07:40.480 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888071 2026-02-13T19:07:40.480 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:40.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888071 2026-02-13T19:07:40.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-13T19:07:40.482 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888071 2026-02-13T19:07:40.482 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:07:40.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888069 -lt 146028888071 2026-02-13T19:07:40.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:07:41.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:07:41.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:07:41.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-13T19:07:41.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:41.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626438 2026-02-13T19:07:41.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:41.938 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:07:41.939 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626438 2026-02-13T19:07:41.939 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:41.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626438 2026-02-13T19:07:41.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626438' 2026-02-13T19:07:41.941 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 180388626438 2026-02-13T19:07:41.941 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:07:42.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626439 -lt 180388626438 2026-02-13T19:07:42.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:07:42.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364806 2026-02-13T19:07:42.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:07:42.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:07:42.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364806 2026-02-13T19:07:42.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:07:42.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364806 2026-02-13T19:07:42.170 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364806' 2026-02-13T19:07:42.171 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 214748364806 2026-02-13T19:07:42.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:07:42.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364806 -lt 214748364806 2026-02-13T19:07:42.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:07:42.399 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:42.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:42.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:07:42.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:07:42.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:07:42.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:07:42.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:07:42.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:07:43.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:07:43.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:07:43.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:07:43.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:07:43.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-13T19:07:43.277 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:07:43.278 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:07:43.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-13T19:07:43.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:07:43.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:07:43.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-13T19:07:43.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:07:43.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:07:43.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-13T19:07:43.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:07:43.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:07:43.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-13T19:07:43.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:07:43.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:07:43.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-13T19:07:43.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:07:43.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:07:43.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-13T19:07:43.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:07:43.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:07:43.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:07:43.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:07:43.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:07:43.631 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:07:43.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:07:43.916 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:07:43.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:07:43.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:07:43.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:07:44.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:07:44.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:07:44.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:07:44.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:07:44.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:07:44.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:07:44.038 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:07:44.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:44.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:07:44.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:07:44.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:44.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:07:44.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:07:44.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:07:44.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:07:44.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:07:44.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:07:44.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:07:44.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:07:44.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:07:44.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:07:44.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:07:44.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:07:44.060 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:07:44.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:07:44.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:07:44.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:07:44.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:07:44.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:07:44.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:44.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:07:44.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:07:44.064 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:07:44.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:07:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:07:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:07:44.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:07:44.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:07:44.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:07:44.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:07:44.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:07:44.068 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:07:44.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:07:44.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:07:44.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:07:44.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:07:44.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:07:44.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:44.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:44.089 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:44.090 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.090 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.090 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:44.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:07:44.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:07:44.114 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:44.115 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:07:44.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:07:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:07:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:07:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:07:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:07:44.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.159 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:44.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:07:44.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:07:44.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:44.301 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:44.302 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:07:44.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:07:44.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:07:44.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:07:44.411 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:07:44.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:07:45.420 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:07:45.420 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:07:45.420 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:07:45.420 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:07:45.420 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:45.421 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:45.421 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:07:45.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:07:45.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:07:45.459 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:07:45.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:07:45.465 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:07:44.105+0000 7f924682cc00 0 load: jerasure load: lrc 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:45.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-13T19:07:45.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-13T19:07:45.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:07:45.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:07:45.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:45.472 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:45.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b56ee6ac-ff23-4398-87e2-7ed28c166b6a 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b56ee6ac-ff23-4398-87e2-7ed28c166b6a' 2026-02-13T19:07:45.474 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 b56ee6ac-ff23-4398-87e2-7ed28c166b6a 2026-02-13T19:07:45.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:45.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBdo9pkVasHBAA0tdU2UlCV6LG22iCeeVrQw== 2026-02-13T19:07:45.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBdo9pkVasHBAA0tdU2UlCV6LG22iCeeVrQw=="}' 2026-02-13T19:07:45.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b56ee6ac-ff23-4398-87e2-7ed28c166b6a -i td/test-erasure-eio/0/new.json 2026-02-13T19:07:45.584 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:45.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:07:45.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBdo9pkVasHBAA0tdU2UlCV6LG22iCeeVrQw== --osd-uuid b56ee6ac-ff23-4398-87e2-7ed28c166b6a 2026-02-13T19:07:45.610 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:45.609+0000 7fcbb7252780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:45.612 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:45.611+0000 7fcbb7252780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:45.613 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:45.612+0000 7fcbb7252780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:45.613 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:45.612+0000 7fcbb7252780 -1 bdev(0x33e0c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:45.613 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:45.612+0000 7fcbb7252780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:07:46.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:07:46.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:46.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:07:46.039 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:07:46.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:46.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:07:46.146 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:07:46.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:46.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:46.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:46.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:46.162 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:46.162+0000 7fa246635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:46.164 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:46.163+0000 7fa246635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:46.165 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:46.164+0000 7fa246635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:46.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:46.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:46.245 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:46.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:46.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:07:46.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:46.385 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:46.385+0000 7fa246635780 -1 Falling back to public interface 2026-02-13T19:07:46.679 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:46.678+0000 7fa246635780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:07:47.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:47.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:47.350 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:47.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:47.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:47.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:07:47.577 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3323900542,v1:127.0.0.1:6803/3323900542] [v2:127.0.0.1:6804/3323900542,v1:127.0.0.1:6805/3323900542] exists,up b56ee6ac-ff23-4398-87e2-7ed28c166b6a 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:47.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:47.582 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:47.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d064d8b-7278-4a83-a1a9-b2617daa1110 2026-02-13T19:07:47.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1d064d8b-7278-4a83-a1a9-b2617daa1110' 2026-02-13T19:07:47.585 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 1d064d8b-7278-4a83-a1a9-b2617daa1110 2026-02-13T19:07:47.585 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:47.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCDdo9p/dNDIxAAIqtMGQvPi3+YD88vTxVZGg== 2026-02-13T19:07:47.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCDdo9p/dNDIxAAIqtMGQvPi3+YD88vTxVZGg=="}' 2026-02-13T19:07:47.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d064d8b-7278-4a83-a1a9-b2617daa1110 -i td/test-erasure-eio/1/new.json 2026-02-13T19:07:47.819 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:47.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:07:47.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDdo9p/dNDIxAAIqtMGQvPi3+YD88vTxVZGg== --osd-uuid 1d064d8b-7278-4a83-a1a9-b2617daa1110 2026-02-13T19:07:47.845 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:47.845+0000 7f433afd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:47.847 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:47.846+0000 7f433afd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:47.848 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:47.847+0000 7f433afd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:47.848 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:47.847+0000 7f433afd8780 -1 bdev(0x3174c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:47.848 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:47.847+0000 7f433afd8780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:07:48.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:07:48.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:48.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:07:48.156 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:07:48.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:48.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:07:48.477 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:07:48.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:48.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:48.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:48.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:48.495 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:48.494+0000 7f9d88a87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:48.496 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:48.495+0000 7f9d88a87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:48.497 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:48.496+0000 7f9d88a87780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:48.628 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:48.627+0000 7f9d88a87780 -1 Falling back to public interface 2026-02-13T19:07:48.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:07:48.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:48.709 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:48.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:48.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:07:48.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:48.939 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:48.938+0000 7f9d88a87780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:07:49.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:49.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:49.933 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:49.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:49.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:49.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:07:50.158 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2862587608,v1:127.0.0.1:6811/2862587608] [v2:127.0.0.1:6812/2862587608,v1:127.0.0.1:6813/2862587608] exists,up 1d064d8b-7278-4a83-a1a9-b2617daa1110 2026-02-13T19:07:50.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:50.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:50.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:50.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:50.164 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:50.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=23caf6a7-ed39-4410-95da-9d2ac889d854 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 23caf6a7-ed39-4410-95da-9d2ac889d854' 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 23caf6a7-ed39-4410-95da-9d2ac889d854 2026-02-13T19:07:50.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:50.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGdo9p3SlHChAA3WH0g9FlqGvFOUZY6T4BXg== 2026-02-13T19:07:50.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGdo9p3SlHChAA3WH0g9FlqGvFOUZY6T4BXg=="}' 2026-02-13T19:07:50.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 23caf6a7-ed39-4410-95da-9d2ac889d854 -i td/test-erasure-eio/2/new.json 2026-02-13T19:07:50.406 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:07:50.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:07:50.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGdo9p3SlHChAA3WH0g9FlqGvFOUZY6T4BXg== --osd-uuid 23caf6a7-ed39-4410-95da-9d2ac889d854 2026-02-13T19:07:50.432 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:50.431+0000 7f42e4829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:50.434 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:50.433+0000 7f42e4829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:50.434 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:50.434+0000 7f42e4829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:50.435 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:50.434+0000 7f42e4829780 -1 bdev(0x33e8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:50.435 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:50.434+0000 7f42e4829780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:07:50.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:07:50.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:50.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:07:50.753 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:07:50.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:51.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:07:51.073 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:07:51.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:51.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:51.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:51.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:51.089 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:51.088+0000 7f92175c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:51.090 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:51.090+0000 7f92175c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:51.091 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:51.090+0000 7f92175c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:51.215 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:51.214+0000 7f92175c6780 -1 Falling back to public interface 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:51.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:51.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:51.301 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:51.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:51.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:07:51.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:51.537 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:51.536+0000 7f92175c6780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:07:52.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:52.532 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:52.532 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:52.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:52.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:52.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:07:52.761 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4138863474,v1:127.0.0.1:6819/4138863474] [v2:127.0.0.1:6820/4138863474,v1:127.0.0.1:6821/4138863474] exists,up 23caf6a7-ed39-4410-95da-9d2ac889d854 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:07:52.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:52.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:52.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c95b9f02-b685-443e-8b29-232fd9afd4c5 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c95b9f02-b685-443e-8b29-232fd9afd4c5' 2026-02-13T19:07:52.769 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 c95b9f02-b685-443e-8b29-232fd9afd4c5 2026-02-13T19:07:52.770 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:52.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCIdo9p4i8/LhAA13KtqHaWt9HJUtaksN8XYQ== 2026-02-13T19:07:52.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCIdo9p4i8/LhAA13KtqHaWt9HJUtaksN8XYQ=="}' 2026-02-13T19:07:52.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c95b9f02-b685-443e-8b29-232fd9afd4c5 -i td/test-erasure-eio/3/new.json 2026-02-13T19:07:53.008 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:07:53.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:07:53.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIdo9p4i8/LhAA13KtqHaWt9HJUtaksN8XYQ== --osd-uuid c95b9f02-b685-443e-8b29-232fd9afd4c5 2026-02-13T19:07:53.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.034+0000 7f54a3a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.036 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.036+0000 7f54a3a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.037 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.036+0000 7f54a3a42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.038 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.037+0000 7f54a3a42780 -1 bdev(0x2944c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:53.038 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.037+0000 7f54a3a42780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:07:53.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:07:53.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:53.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:07:53.367 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:07:53.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:53.694 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:07:53.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:07:53.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:53.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:53.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:53.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:53.712 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.712+0000 7f7de983a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.714 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.713+0000 7f7de983a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.715 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.714+0000 7f7de983a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:53.852 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:53.851+0000 7f7de983a780 -1 Falling back to public interface 2026-02-13T19:07:53.922 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:53.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:53.924 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:53.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:53.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:07:54.145 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:54.143+0000 7f7de983a780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:07:54.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:55.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:55.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:55.150 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:55.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:55.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:55.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:07:55.384 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1439294824,v1:127.0.0.1:6827/1439294824] [v2:127.0.0.1:6828/1439294824,v1:127.0.0.1:6829/1439294824] exists,up c95b9f02-b685-443e-8b29-232fd9afd4c5 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:55.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:55.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:55.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4aa0782d-542a-4bfb-9784-f8bb619a7e76 2026-02-13T19:07:55.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 4aa0782d-542a-4bfb-9784-f8bb619a7e76' 2026-02-13T19:07:55.392 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 4aa0782d-542a-4bfb-9784-f8bb619a7e76 2026-02-13T19:07:55.392 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:55.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLdo9pT6e9FxAAinPGvtmL4c4WvQYl1Ofd1A== 2026-02-13T19:07:55.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLdo9pT6e9FxAAinPGvtmL4c4WvQYl1Ofd1A=="}' 2026-02-13T19:07:55.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4aa0782d-542a-4bfb-9784-f8bb619a7e76 -i td/test-erasure-eio/4/new.json 2026-02-13T19:07:55.635 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:07:55.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:07:55.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLdo9pT6e9FxAAinPGvtmL4c4WvQYl1Ofd1A== --osd-uuid 4aa0782d-542a-4bfb-9784-f8bb619a7e76 2026-02-13T19:07:55.661 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:55.659+0000 7fb87ac31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:55.662 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:55.661+0000 7fb87ac31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:55.663 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:55.662+0000 7fb87ac31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:55.664 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:55.662+0000 7fb87ac31780 -1 bdev(0x28eac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:55.664 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:55.662+0000 7fb87ac31780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:07:55.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:07:55.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:55.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:07:55.984 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:07:55.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:56.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:07:56.308 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:07:56.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:56.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:56.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:56.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:56.324 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:56.323+0000 7fbee2635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:56.326 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:56.324+0000 7fbee2635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:56.327 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:56.325+0000 7fbee2635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:56.452 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:56.450+0000 7fbee2635780 -1 Falling back to public interface 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:56.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:07:56.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:56.811 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:56.809+0000 7fbee2635780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:57.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:07:58.000 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1507386973,v1:127.0.0.1:6835/1507386973] [v2:127.0.0.1:6836/1507386973,v1:127.0.0.1:6837/1507386973] exists,up 4aa0782d-542a-4bfb-9784-f8bb619a7e76 2026-02-13T19:07:58.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:07:58.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:07:58.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:07:58.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:07:58.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-13T19:07:58.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:07:58.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:07:58.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:07:58.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:07:58.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-13T19:07:58.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:07:58.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:07:58.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:07:58.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:07:58.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-13T19:07:58.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-13T19:07:58.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:07:58.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:07:58.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:07:58.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:07:58.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:07:58.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-13T19:07:58.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:07:58.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24d63ca4-339a-40af-9f85-dbb5603f7c94 2026-02-13T19:07:58.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 24d63ca4-339a-40af-9f85-dbb5603f7c94' 2026-02-13T19:07:58.018 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 24d63ca4-339a-40af-9f85-dbb5603f7c94 2026-02-13T19:07:58.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:07:58.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOdo9powHaABAAz+u4QeVfcnn1/HjziP3D6A== 2026-02-13T19:07:58.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOdo9powHaABAAz+u4QeVfcnn1/HjziP3D6A=="}' 2026-02-13T19:07:58.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24d63ca4-339a-40af-9f85-dbb5603f7c94 -i td/test-erasure-eio/5/new.json 2026-02-13T19:07:58.248 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:07:58.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-13T19:07:58.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOdo9powHaABAAz+u4QeVfcnn1/HjziP3D6A== --osd-uuid 24d63ca4-339a-40af-9f85-dbb5603f7c94 2026-02-13T19:07:58.274 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.272+0000 7fee50e53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:58.275 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.274+0000 7fee50e53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:58.276 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.274+0000 7fee50e53780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:58.277 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.275+0000 7fee50e53780 -1 bdev(0x3360c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:07:58.277 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.275+0000 7fee50e53780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-13T19:07:58.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-13T19:07:58.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:07:58.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:07:58.592 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:07:58.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:07:58.933 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:07:58.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:07:58.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:07:58.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:07:58.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:07:58.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:07:58.950 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.948+0000 7f2fb5433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:58.951 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.950+0000 7f2fb5433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:58.952 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:58.951+0000 7f2fb5433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:07:59.109 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:59.106+0000 7f2fb5433780 -1 Falling back to public interface 2026-02-13T19:07:59.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:07:59.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:07:59.163 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:07:59.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:07:59.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:07:59.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:07:59.448 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:07:59.446+0000 7f2fb5433780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:08:00.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:00.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:00.388 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:00.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:00.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:00.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:08:00.616 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2991713774,v1:127.0.0.1:6843/2991713774] [v2:127.0.0.1:6844/2991713774,v1:127.0.0.1:6845/2991713774] exists,up 24d63ca4-339a-40af-9f85-dbb5603f7c94 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-13T19:08:00.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:00.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-13T19:08:00.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:00.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b8e9ad27-960e-49f2-8b29-80bea6463e99 2026-02-13T19:08:00.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 b8e9ad27-960e-49f2-8b29-80bea6463e99' 2026-02-13T19:08:00.620 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 b8e9ad27-960e-49f2-8b29-80bea6463e99 2026-02-13T19:08:00.620 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:00.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQdo9pIuCjJRAAsBkN7uc36qwk2qj8ZdLC9w== 2026-02-13T19:08:00.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQdo9pIuCjJRAAsBkN7uc36qwk2qj8ZdLC9w=="}' 2026-02-13T19:08:00.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b8e9ad27-960e-49f2-8b29-80bea6463e99 -i td/test-erasure-eio/6/new.json 2026-02-13T19:08:00.864 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:08:00.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-13T19:08:00.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCQdo9pIuCjJRAAsBkN7uc36qwk2qj8ZdLC9w== --osd-uuid b8e9ad27-960e-49f2-8b29-80bea6463e99 2026-02-13T19:08:00.890 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:00.888+0000 7f22fbc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:00.891 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:00.890+0000 7f22fbc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:00.892 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:00.891+0000 7f22fbc30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:00.893 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:00.891+0000 7f22fbc30780 -1 bdev(0x3c48c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:00.893 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:00.891+0000 7f22fbc30780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-13T19:08:01.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-13T19:08:01.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:01.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:08:01.229 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:08:01.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:01.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:08:01.552 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:08:01.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:01.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:01.569 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:01.567+0000 7f38bc827780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:01.570 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:01.568+0000 7f38bc827780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:01.571 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:01.569+0000 7f38bc827780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:01.714 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:01.712+0000 7f38bc827780 -1 Falling back to public interface 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:01.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:01.781 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:01.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:01.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:08:02.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:02.076 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:02.074+0000 7f38bc827780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:08:03.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:03.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:03.007 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:03.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:03.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:03.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:08:03.235 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2094962383,v1:127.0.0.1:6851/2094962383] [v2:127.0.0.1:6852/2094962383,v1:127.0.0.1:6853/2094962383] exists,up b8e9ad27-960e-49f2-8b29-80bea6463e99 2026-02-13T19:08:03.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:03.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:03.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:03.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:08:03.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:08:03.282 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:08:03.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:08:03.289 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:07:46.387+0000 7fa246635780 0 load: jerasure load: lrc 2026-02-13T19:08:03.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:08:03.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-13T19:08:03.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:08:03.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:08:03.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:08:03.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:08:03.947 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:08:03.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:08:04.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:08:04.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:08:04.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:08:04.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:08:04.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:08:05.046 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:08:05.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:08:05.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:08:05.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-13T19:08:05.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-13T19:08:05.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-13T19:08:05.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.336 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:08:05.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-13T19:08:05.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-13T19:08:05.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-13T19:08:05.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:08:05.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-13T19:08:05.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-13T19:08:05.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629' 2026-02-13T19:08:05.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.466 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:08:05.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-13T19:08:05.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-13T19:08:05.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-111669149701' 2026-02-13T19:08:05.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.529 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:08:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-13T19:08:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-13T19:08:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-111669149701 4-150323855364' 2026-02-13T19:08:05.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:08:05.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-13T19:08:05.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-13T19:08:05.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-111669149701 4-150323855364 5-184683593732' 2026-02-13T19:08:05.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:05.659 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:08:05.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397508 2026-02-13T19:08:05.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397508 2026-02-13T19:08:05.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-111669149701 4-150323855364 5-184683593732 6-210453397508' 2026-02-13T19:08:05.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:05.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-13T19:08:05.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:05.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:08:05.724 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-13T19:08:05.724 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:05.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-13T19:08:05.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-13T19:08:05.726 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803782 2026-02-13T19:08:05.726 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:05.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-13T19:08:05.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:08:06.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:08:06.954 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:07.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-13T19:08:07.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:07.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-13T19:08:07.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:07.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:08:07.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-13T19:08:07.179 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:07.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-13T19:08:07.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-13T19:08:07.180 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574854 2026-02-13T19:08:07.181 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:08:07.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-13T19:08:07.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:07.408 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378629 2026-02-13T19:08:07.408 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:07.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:08:07.409 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-13T19:08:07.409 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:07.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378629 2026-02-13T19:08:07.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-13T19:08:07.410 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378629 2026-02-13T19:08:07.411 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:08:07.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-13T19:08:07.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:07.636 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149701 2026-02-13T19:08:07.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:07.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:08:07.638 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-13T19:08:07.638 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:07.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149701 2026-02-13T19:08:07.639 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149701 2026-02-13T19:08:07.639 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-13T19:08:07.639 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:08:07.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-13T19:08:07.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:07.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855364 2026-02-13T19:08:07.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:07.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:08:07.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-13T19:08:07.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:07.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855364 2026-02-13T19:08:07.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-13T19:08:07.871 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855364 2026-02-13T19:08:07.871 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:08:08.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-13T19:08:08.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:08.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593732 2026-02-13T19:08:08.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:08.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:08:08.101 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-13T19:08:08.101 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:08.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-13T19:08:08.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-13T19:08:08.103 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593732 2026-02-13T19:08:08.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:08:08.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-13T19:08:08.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:08.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397508 2026-02-13T19:08:08.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:08.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:08:08.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397508 2026-02-13T19:08:08.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:08.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397508 2026-02-13T19:08:08.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397508' 2026-02-13T19:08:08.331 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 210453397508 2026-02-13T19:08:08.331 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:08:08.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397509 -lt 210453397508 2026-02-13T19:08:08.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:08:08.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:08.556 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:08.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:08:08.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:08:08.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:08:08.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:08:08.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:08:08.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:08:08.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:08:08.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:08:09.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:08:09.086 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:08:09.086 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:09.086 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:08:09.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:08:09.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:08:09.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:08:09.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:08:09.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:08:09.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:08:09.481 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:08:09.709 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-13T19:08:09.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-13T19:08:09.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:08:09.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:08:09.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:08:09.710 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:08:09.712 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:08:09.713 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:08:09.758 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:08:09.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:08:09.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:08:09.760 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:08:09.760 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:08:09.760 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:08:09.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-13T19:08:09.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-13T19:08:10.089 INFO:tasks.workunit.client.0.trial034.stderr:osd.0 is already out. 2026-02-13T19:08:10.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-13T19:08:10.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:08:10.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:08:10.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:08:10.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:08:10.099 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:08:10.099 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:08:10.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:08:10.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:08:10.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:08:10.186 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:08:10.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:08:10.411 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:10.411 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:08:10.411 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:08:10.411 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:08:10.412 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:08:10.412 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:08:10.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:08:10.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:08:10.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-13T19:08:10.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-13T19:08:10.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-13T19:08:10.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.475 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:08:10.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-13T19:08:10.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-13T19:08:10.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-13T19:08:10.539 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.539 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:08:10.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378632 2026-02-13T19:08:10.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378632 2026-02-13T19:08:10.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632' 2026-02-13T19:08:10.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:08:10.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-13T19:08:10.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-13T19:08:10.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149704' 2026-02-13T19:08:10.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.668 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:08:10.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-13T19:08:10.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-13T19:08:10.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149704 4-150323855367' 2026-02-13T19:08:10.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:08:10.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-13T19:08:10.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-13T19:08:10.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149704 4-150323855367 5-184683593735' 2026-02-13T19:08:10.797 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:10.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:08:10.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397511 2026-02-13T19:08:10.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397511 2026-02-13T19:08:10.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-111669149704 4-150323855367 5-184683593735 6-210453397511' 2026-02-13T19:08:10.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:10.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-13T19:08:10.862 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:10.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:08:10.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-13T19:08:10.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:10.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-13T19:08:10.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-13T19:08:10.865 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803785 2026-02-13T19:08:10.865 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:11.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 0 -lt 25769803785 2026-02-13T19:08:11.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:08:12.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:08:12.092 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:12.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 0 -lt 25769803785 2026-02-13T19:08:12.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:08:13.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:08:13.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:13.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-13T19:08:13.554 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:13.554 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-13T19:08:13.554 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:13.555 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:08:13.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-13T19:08:13.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:13.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-13T19:08:13.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-13T19:08:13.557 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574857 2026-02-13T19:08:13.557 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:08:13.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-13T19:08:13.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:13.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378632 2026-02-13T19:08:13.786 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:13.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:08:13.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378632 2026-02-13T19:08:13.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:13.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378632 2026-02-13T19:08:13.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378632' 2026-02-13T19:08:13.788 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378632 2026-02-13T19:08:13.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:08:14.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-13T19:08:14.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:14.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149704 2026-02-13T19:08:14.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:14.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:08:14.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-13T19:08:14.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:14.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149704 2026-02-13T19:08:14.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-13T19:08:14.020 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149704 2026-02-13T19:08:14.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:08:14.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-13T19:08:14.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:14.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855367 2026-02-13T19:08:14.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:14.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:08:14.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-13T19:08:14.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:14.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855367 2026-02-13T19:08:14.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-13T19:08:14.250 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 150323855367 2026-02-13T19:08:14.250 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:08:14.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855368 -lt 150323855367 2026-02-13T19:08:14.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:14.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-13T19:08:14.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:14.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:08:14.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-13T19:08:14.477 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:14.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-13T19:08:14.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-13T19:08:14.479 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593735 2026-02-13T19:08:14.479 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:08:14.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-13T19:08:14.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:14.706 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397511 2026-02-13T19:08:14.706 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:14.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:08:14.708 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397511 2026-02-13T19:08:14.708 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:14.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397511 2026-02-13T19:08:14.709 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 210453397511 2026-02-13T19:08:14.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397511' 2026-02-13T19:08:14.709 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:08:14.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397512 -lt 210453397511 2026-02-13T19:08:14.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:08:14.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:14.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:15.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:08:15.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:08:15.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:08:15.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:08:15.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:08:15.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:08:15.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:08:15.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:08:15.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:08:15.482 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:08:15.482 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:15.482 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:15.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:08:15.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:08:15.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-13T19:08:15.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:08:15.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:08:15.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-13T19:08:15.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:08:15.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:08:15.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-13T19:08:15.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:08:15.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:08:15.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-13T19:08:15.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:08:15.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:08:15.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-13T19:08:15.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:08:15.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:08:15.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:08:15.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:08:15.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:08:16.135 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:08:16.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:08:16.422 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:08:16.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:08:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:08:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:08:16.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:08:16.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:08:16.542 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:08:16.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:08:16.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:08:16.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:08:16.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:08:16.545 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:08:16.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:08:16.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:08:16.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:08:16.547 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:08:16.547 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:08:16.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:08:16.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:08:16.563 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:08:16.563 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.563 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:08:16.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:08:16.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:08:16.567 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:08:16.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:08:16.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:08:16.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:08:16.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:08:16.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:08:16.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:08:16.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:08:16.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:08:16.570 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:08:16.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:08:16.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:08:16.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:08:16.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:08:16.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:08:16.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:08:16.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:08:16.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:08:16.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:08:16.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:08:16.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:08:16.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:08:16.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.596 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.597 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:16.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:08:16.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:08:16.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:08:16.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:08:16.621 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:08:16.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:08:16.622 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.623 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:08:16.624 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:08:16.624 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:08:16.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:08:16.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:08:16.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:08:16.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:08:16.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:08:16.666 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:08:16.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:08:16.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:08:16.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:08:16.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:08:16.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:08:16.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:08:16.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:08:16.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:08:16.808 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:16.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:16.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:08:16.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:08:16.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:08:16.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:08:16.922 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:08:16.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:17.931 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:17.932 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:08:17.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:08:17.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:08:17.971 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:08:17.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:08:17.977 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:08:16.613+0000 7f5c3842ec00 0 load: jerasure load: lrc 2026-02-13T19:08:17.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-13T19:08:17.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-13T19:08:17.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:08:17.980 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:17.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:17.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:17.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=79a1d5b9-cc9f-44ba-ab33-de34c1acb086 2026-02-13T19:08:17.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 79a1d5b9-cc9f-44ba-ab33-de34c1acb086' 2026-02-13T19:08:17.988 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 79a1d5b9-cc9f-44ba-ab33-de34c1acb086 2026-02-13T19:08:17.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:17.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChdo9pI7dBOxAAv2VEn9AH3KIpi7NAf3gyMQ== 2026-02-13T19:08:17.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChdo9pI7dBOxAAv2VEn9AH3KIpi7NAf3gyMQ=="}' 2026-02-13T19:08:17.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 79a1d5b9-cc9f-44ba-ab33-de34c1acb086 -i td/test-erasure-eio/0/new.json 2026-02-13T19:08:18.102 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:18.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:08:18.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChdo9pI7dBOxAAv2VEn9AH3KIpi7NAf3gyMQ== --osd-uuid 79a1d5b9-cc9f-44ba-ab33-de34c1acb086 2026-02-13T19:08:18.129 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.128+0000 7f9ea1861780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.131 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.130+0000 7f9ea1861780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.131+0000 7f9ea1861780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.131+0000 7f9ea1861780 -1 bdev(0x3f82c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:18.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.131+0000 7f9ea1861780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:08:18.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:08:18.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:18.520 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:08:18.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:08:18.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:18.628 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:08:18.629 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:08:18.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:18.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:18.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:18.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:18.648 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.647+0000 7f33a5042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.649 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.649+0000 7f33a5042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.650 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.649+0000 7f33a5042780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:18.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:18.730 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:18.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:18.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:08:18.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:18.846 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:18.845+0000 7f33a5042780 -1 Falling back to public interface 2026-02-13T19:08:19.123 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:19.122+0000 7f33a5042780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:08:19.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:19.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:19.829 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:19.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:19.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:19.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:08:20.057 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2229092877,v1:127.0.0.1:6803/2229092877] [v2:127.0.0.1:6804/2229092877,v1:127.0.0.1:6805/2229092877] exists,up 79a1d5b9-cc9f-44ba-ab33-de34c1acb086 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:20.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:20.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:20.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b46d014-01e5-4b5f-8d4b-94cfc33dd36f 2026-02-13T19:08:20.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1b46d014-01e5-4b5f-8d4b-94cfc33dd36f' 2026-02-13T19:08:20.065 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 1b46d014-01e5-4b5f-8d4b-94cfc33dd36f 2026-02-13T19:08:20.065 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:20.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCkdo9pj3RcBBAAXbz5HmEYlau/TEScJZvubQ== 2026-02-13T19:08:20.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCkdo9pj3RcBBAAXbz5HmEYlau/TEScJZvubQ=="}' 2026-02-13T19:08:20.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b46d014-01e5-4b5f-8d4b-94cfc33dd36f -i td/test-erasure-eio/1/new.json 2026-02-13T19:08:20.311 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:20.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:08:20.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCkdo9pj3RcBBAAXbz5HmEYlau/TEScJZvubQ== --osd-uuid 1b46d014-01e5-4b5f-8d4b-94cfc33dd36f 2026-02-13T19:08:20.340 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:20.340+0000 7f7c91a3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:20.342 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:20.341+0000 7f7c91a3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:20.343 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:20.342+0000 7f7c91a3f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:20.343 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:20.342+0000 7f7c91a3f780 -1 bdev(0x2a1ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:20.343 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:20.342+0000 7f7c91a3f780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:08:20.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:08:20.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:20.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:08:20.680 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:08:20.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:21.004 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:08:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:08:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:21.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:21.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:21.022+0000 7f261e439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:21.024 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:21.024+0000 7f261e439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:21.025 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:21.024+0000 7f261e439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:21.154 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:21.154+0000 7f261e439780 -1 Falling back to public interface 2026-02-13T19:08:21.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:08:21.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:21.242 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:21.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:21.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:08:21.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:21.507 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:21.506+0000 7f261e439780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:08:22.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:22.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:22.481 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:22.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:22.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:22.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:08:22.711 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2643283748,v1:127.0.0.1:6811/2643283748] [v2:127.0.0.1:6812/2643283748,v1:127.0.0.1:6813/2643283748] exists,up 1b46d014-01e5-4b5f-8d4b-94cfc33dd36f 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:22.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:22.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:22.717 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 2026-02-13T19:08:22.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54' 2026-02-13T19:08:22.719 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 2026-02-13T19:08:22.719 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmdo9pqpxKKxAAyiZXJZrAUVDJ+acL10flfg== 2026-02-13T19:08:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmdo9pqpxKKxAAyiZXJZrAUVDJ+acL10flfg=="}' 2026-02-13T19:08:22.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 -i td/test-erasure-eio/2/new.json 2026-02-13T19:08:22.957 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:08:22.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:08:22.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmdo9pqpxKKxAAyiZXJZrAUVDJ+acL10flfg== --osd-uuid 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 2026-02-13T19:08:22.986 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:22.985+0000 7f095bfd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:22.987 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:22.987+0000 7f095bfd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:22.988 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:22.987+0000 7f095bfd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:22.989 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:22.988+0000 7f095bfd6780 -1 bdev(0x36a8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:22.989 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:22.988+0000 7f095bfd6780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:08:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:08:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:23.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:08:23.311 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:08:23.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:23.632 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:08:23.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:08:23.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:23.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:23.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:23.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:23.652 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:23.651+0000 7f7e5502c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:23.653 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:23.653+0000 7f7e5502c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:23.654 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:23.654+0000 7f7e5502c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:23.787 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:23.786+0000 7f7e5502c780 -1 Falling back to public interface 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:23.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:23.861 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:23.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:23.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:08:24.070 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:24.069+0000 7f7e5502c780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:08:24.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:25.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:25.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:25.109 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:25.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:25.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:25.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:08:25.341 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1388365980,v1:127.0.0.1:6819/1388365980] [v2:127.0.0.1:6820/1388365980,v1:127.0.0.1:6821/1388365980] exists,up 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:25.342 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:25.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:08:25.345 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:25.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dd2a2b80-ec32-474c-8cec-8a685c01134d 2026-02-13T19:08:25.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 dd2a2b80-ec32-474c-8cec-8a685c01134d' 2026-02-13T19:08:25.346 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 dd2a2b80-ec32-474c-8cec-8a685c01134d 2026-02-13T19:08:25.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:25.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCpdo9pL2ZJFRAA5Yb08gyg/ppGyJQ1xPIKLA== 2026-02-13T19:08:25.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCpdo9pL2ZJFRAA5Yb08gyg/ppGyJQ1xPIKLA=="}' 2026-02-13T19:08:25.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dd2a2b80-ec32-474c-8cec-8a685c01134d -i td/test-erasure-eio/3/new.json 2026-02-13T19:08:25.591 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:08:25.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:08:25.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpdo9pL2ZJFRAA5Yb08gyg/ppGyJQ1xPIKLA== --osd-uuid dd2a2b80-ec32-474c-8cec-8a685c01134d 2026-02-13T19:08:25.621 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:25.620+0000 7fbe4ee10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:25.622 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:25.621+0000 7fbe4ee10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:25.623 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:25.622+0000 7fbe4ee10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:25.623 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:25.622+0000 7fbe4ee10780 -1 bdev(0x23eac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:25.623 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:25.622+0000 7fbe4ee10780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:08:25.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:08:25.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:25.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:08:25.944 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:08:25.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:26.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:08:26.272 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:08:26.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:26.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:26.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:26.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:26.291 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:26.290+0000 7f5441c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:26.292 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:26.291+0000 7f5441c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:26.293 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:26.292+0000 7f5441c42780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:26.412 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:26.412+0000 7f5441c42780 -1 Falling back to public interface 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:26.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:26.505 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:26.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:26.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:08:26.698 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:26.697+0000 7f5441c42780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:08:26.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:27.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:27.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:27.733 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:27.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:27.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:27.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:08:27.966 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1375743155,v1:127.0.0.1:6827/1375743155] [v2:127.0.0.1:6828/1375743155,v1:127.0.0.1:6829/1375743155] exists,up dd2a2b80-ec32-474c-8cec-8a685c01134d 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:27.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:08:27.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:08:27.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c 2026-02-13T19:08:27.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c' 2026-02-13T19:08:27.974 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c 2026-02-13T19:08:27.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:08:27.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrdo9pVqmGOhAApn4YA8nYmmORs303QR/M4Q== 2026-02-13T19:08:27.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrdo9pVqmGOhAApn4YA8nYmmORs303QR/M4Q=="}' 2026-02-13T19:08:27.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c -i td/test-erasure-eio/4/new.json 2026-02-13T19:08:28.216 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:08:28.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:08:28.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCrdo9pVqmGOhAApn4YA8nYmmORs303QR/M4Q== --osd-uuid 562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c 2026-02-13T19:08:28.244 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.243+0000 7fe2b3235780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:28.246 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.245+0000 7fe2b3235780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:28.247 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.246+0000 7fe2b3235780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:28.247 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.246+0000 7fe2b3235780 -1 bdev(0x23f8c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:08:28.247 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.246+0000 7fe2b3235780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:08:28.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:08:28.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:08:28.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:08:28.565 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:08:28.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:08:28.915 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:08:28.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:08:28.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:28.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:08:28.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:08:28.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:08:28.934 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.933+0000 7f636180f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:28.936 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.935+0000 7f636180f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:28.936 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:28.936+0000 7f636180f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:29.073 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:29.073+0000 7f636180f780 -1 Falling back to public interface 2026-02-13T19:08:29.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:29.149 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:29.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:29.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:08:29.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:29.412 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:29.411+0000 7f636180f780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:08:30.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:30.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:30.382 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:30.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:30.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:30.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:08:30.609 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1798746561,v1:127.0.0.1:6835/1798746561] [v2:127.0.0.1:6836/1798746561,v1:127.0.0.1:6837/1798746561] exists,up 562ef0dc-6e8d-4dd4-88bf-723e7f7fa60c 2026-02-13T19:08:30.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:30.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:08:30.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:08:30.656 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:08:30.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:08:30.663 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:08:18.848+0000 7f33a5042780 0 load: jerasure load: lrc 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:08:30.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:08:30.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:08:30.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:08:30.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:08:30.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:08:30.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:08:31.325 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:08:31.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:08:32.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:08:32.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:08:32.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:08:32.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:08:32.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:08:32.336 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:08:32.336 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:08:32.336 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:08:32.336 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:08:32.336 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:08:32.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:08:32.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:08:32.651 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr:4' 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:32.652 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:08:32.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-13T19:08:32.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-13T19:08:32.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-13T19:08:32.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:32.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:08:32.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-13T19:08:32.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-13T19:08:32.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558' 2026-02-13T19:08:32.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:32.778 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:08:32.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-13T19:08:32.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-13T19:08:32.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411332' 2026-02-13T19:08:32.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:32.843 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:08:32.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-13T19:08:32.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-13T19:08:32.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411332 3-107374182405' 2026-02-13T19:08:32.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:08:32.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:08:32.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-13T19:08:32.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-13T19:08:32.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607558 2-77309411332 3-107374182405 4-146028888068' 2026-02-13T19:08:32.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:32.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-13T19:08:32.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:32.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:08:32.972 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-13T19:08:32.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:32.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-13T19:08:32.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-13T19:08:32.974 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803781 2026-02-13T19:08:32.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:08:33.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-13T19:08:33.201 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:33.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-13T19:08:33.202 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:33.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:08:33.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-13T19:08:33.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:33.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-13T19:08:33.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-13T19:08:33.205 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607558 2026-02-13T19:08:33.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:08:33.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-13T19:08:33.435 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:33.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-13T19:08:33.436 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:33.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:08:33.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-13T19:08:33.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:33.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-13T19:08:33.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-13T19:08:33.438 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411332 2026-02-13T19:08:33.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:08:33.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411330 -lt 77309411332 2026-02-13T19:08:33.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:08:34.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:08:34.662 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:08:34.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-13T19:08:34.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:34.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182405 2026-02-13T19:08:34.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:34.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:08:34.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-13T19:08:34.892 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:34.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182405 2026-02-13T19:08:34.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-13T19:08:34.893 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182405 2026-02-13T19:08:34.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:08:35.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-13T19:08:35.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:08:35.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888068 2026-02-13T19:08:35.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:08:35.121 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:08:35.121 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-13T19:08:35.122 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:08:35.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888068 2026-02-13T19:08:35.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-13T19:08:35.123 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888068 2026-02-13T19:08:35.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:08:35.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-13T19:08:35.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:08:35.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:35.347 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:08:35.669 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:08:35.670 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:08:35.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:08:35.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:08:35.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:08:35.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:08:36.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:08:36.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:08:36.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:08:36.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-13T19:08:36.415 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:32.026314+0000 0'0 40:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-13T19:08:31.260267+0000 0'0 2026-02-13T19:08:31.260267+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:38:52.713766+0000 0 0 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.701526+0000 0'0 40:18 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T21:03:49.334611+0000 0 0 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.524772+0000 0'0 40:90 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T23:24:27.139562+0000 0 0 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.701396+0000 0'0 40:18 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-15T03:12:45.341068+0000 0 0 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:25.627796+0000 0'0 40:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T19:46:13.334858+0000 0 0 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:08:36.416 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:08:36.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:08:36.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:08:36.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:36.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-13T19:08:36.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:08:36.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:08:36.443 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:08:36.443 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:08:36.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:08:36.678 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:08:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:08:36.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:08:36.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:08:36.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:08:36.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-13T19:08:37.214 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already down. 2026-02-13T19:08:37.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-13T19:08:37.466 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already out. 2026-02-13T19:08:37.474 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-13T19:08:37.690 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:08:37.690 INFO:tasks.workunit.client.0.trial034.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:32.026314+0000 0'0 40:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-13T19:08:31.260267+0000 0'0 2026-02-13T19:08:31.260267+0000 0 0 periodic scrub scheduled @ 2026-02-15T01:38:52.713766+0000 0 0 2026-02-13T19:08:37.690 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.701526+0000 0'0 41:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T21:03:49.334611+0000 0 0 2026-02-13T19:08:37.690 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.524772+0000 0'0 41:92 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T23:24:27.139562+0000 0 0 2026-02-13T19:08:37.691 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.701396+0000 0'0 41:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-15T03:12:45.341068+0000 0 0 2026-02-13T19:08:37.691 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:25.627796+0000 0'0 40:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T19:46:13.334858+0000 0 0 2026-02-13T19:08:37.691 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:08:37.691 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:08:37.691 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:08:37.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-13T19:08:37.698 INFO:tasks.workunit.client.0.trial034.stderr:4+0 records in 2026-02-13T19:08:37.698 INFO:tasks.workunit.client.0.trial034.stderr:4+0 records out 2026-02-13T19:08:37.698 INFO:tasks.workunit.client.0.trial034.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.6704e-05 s, 61.4 MB/s 2026-02-13T19:08:37.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-13T19:08:37.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.832 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.832 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.881 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.881 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.898 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.961 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.977 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.090 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.091 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.124 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.268 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.334 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.447 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.530 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.546 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.895 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.946 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:38.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:38.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.047 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.047 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.199 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:08:39.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-13T19:08:39.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-13T19:08:39.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:08:39.350 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-13T19:08:39.351 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:08:39.352 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-13T19:08:39.352 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-13T19:08:39.352 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr:2147483647' 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-13T19:08:39.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-13T19:08:39.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:08:39.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:08:39.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:08:39.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:08:39.583 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:39.584 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:39.584 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:39.584 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:08:39.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:08:39.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:08:39.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:08:39.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:39.632 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:08:39.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:08:39.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-13T19:08:39.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:08:39.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-13T19:08:39.682 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:08:39.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:2147483647' 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:08:39.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-13T19:08:39.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:08:39.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-13T19:08:39.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:08:39.915 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:08:39.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:08:39.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:08:39.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:08:39.964 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-13T19:08:39.964 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:08:39.965 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:08:39.965 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:08:39.966 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:39.966 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:39.966 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:08:39.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:08:39.966 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-13T19:08:40.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-13T19:08:40.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:08:40.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:08:40.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:08:40.018 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:08:40.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:08:40.019 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:08:40.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:08:40.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:08:40.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:08:40.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:08:40.034 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:40.033+0000 7f3200e5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:40.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:40.034+0000 7f3200e5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:40.036 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:40.035+0000 7f3200e5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:08:40.191 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:40.190+0000 7f3200e5d780 -1 Falling back to public interface 2026-02-13T19:08:40.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:08:40.246 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:08:40.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:40.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:08:40.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:08:40.516 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:08:40.515+0000 7f3200e5d780 -1 osd.2 41 log_to_monitors true 2026-02-13T19:08:41.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:08:41.476 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:08:41.477 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:08:41.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:08:41.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:08:41.478 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:08:41.706 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up out weight 0 up_from 47 up_thru 19 down_at 42 last_clean_interval [18,41) [v2:127.0.0.1:6818/74897991,v1:127.0.0.1:6819/74897991] [v2:127.0.0.1:6820/74897991,v1:127.0.0.1:6821/74897991] exists,up 39fb8dc3-32ef-46b7-bcd9-02e2a8431c54 2026-02-13T19:08:41.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:08:41.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:08:41.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:08:41.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-13T19:08:41.949 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already in. 2026-02-13T19:08:41.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-13T19:08:56.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-13T19:08:56.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-13T19:08:56.961 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-13T19:08:56.961 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-13T19:08:56.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-13T19:08:56.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-13T19:08:56.962 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-13T19:08:57.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-13T19:08:57.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-13T19:08:57.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-13T19:08:57.185 INFO:tasks.workunit.client.0.trial034.stdout:active+recovery_unfound+degraded 2026-02-13T19:08:57.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-13T19:08:57.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-13T19:08:57.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-13T19:08:57.397 INFO:tasks.workunit.client.0.trial034.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-13T19:08:57.397 INFO:tasks.workunit.client.0.trial034.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-13T19:08:41.985662+0000 45'101 52:360 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-13T19:08:31.260267+0000 0'0 2026-02-13T19:08:31.260267+0000 0 0 no scrub is scheduled 0 0 2026-02-13T19:08:57.397 INFO:tasks.workunit.client.0.trial034.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:41.984098+0000 0'0 52:59 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T23:18:31.153804+0000 0 0 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.524772+0000 0'0 52:111 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-14T23:24:27.139562+0000 0 0 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:30.701396+0000 0'0 52:41 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-15T03:12:45.341068+0000 0 0 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-13T19:08:41.984144+0000 0'0 52:102 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-13T19:08:16.902827+0000 0'0 2026-02-13T19:08:16.902827+0000 0 0 periodic scrub scheduled @ 2026-02-15T06:31:19.280737+0000 0 0 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stdout: 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-13T19:08:57.398 INFO:tasks.workunit.client.0.trial034.stderr:dumped pgs 2026-02-13T19:08:57.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout:{ 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "num_missing": 1, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "num_unfound": 1, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "objects": [ 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "oid": { 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "oid": "obj75", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "key": "", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "snapid": -2, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "hash": 3565005477, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "max": 0, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "pool": 2, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "namespace": "" 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "need": "45'76", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "have": "0'0", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "flags": "none", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "locations": [ 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: ] 2026-02-13T19:08:57.467 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout: "state": "NotRecovering", 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout: "available_might_have_unfound": true, 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout: "might_have_unfound": [], 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout: "more": false 2026-02-13T19:08:57.468 INFO:tasks.workunit.client.0.trial034.stdout:} 2026-02-13T19:08:57.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout:{ 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: "snap_trimq": "[]", 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: "snap_trimq_len": 0, 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+recovery_unfound+degraded", 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: "epoch": 52, 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.534 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.535 INFO:tasks.workunit.client.0.trial034.stdout: "acting_recovery_backfill": [ 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "2(4)", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "info": { 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "0", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'101", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'75", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "0'0", 2026-02-13T19:08:57.536 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 101, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "51", 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 51, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:08:57.537 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 51, 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 51, 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:08:57.538 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'101", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 361, 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 52, 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+recovery_unfound+degraded", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:08:36.440408+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:08:41.983957+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:08:41.983957+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:08:41.982444+0000", 2026-02-13T19:08:57.539 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:08:41.985662+0000", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 51, 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "0'0", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "0'0", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.540 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 0, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:08:57.541 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 413696, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 101, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 505, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 1, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 1, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 3, 2026-02-13T19:08:57.542 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 1, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 101, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 101, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 404, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:08:57.543 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:08:57.544 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.545 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:08:57.546 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 100 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),4(2)", 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 1 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:08:57.547 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "peer_info": [ 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "0(3)", 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s3", 2026-02-13T19:08:57.548 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "3", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'101", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'101", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "0'0", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 101, 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.549 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 51, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 51, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 51, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.550 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'101", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 349, 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 51, 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:08:36.440408+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.551 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:08:36.859594+0000", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:08:36.859538+0000", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 51, 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "0'0", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "0'0", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.552 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 0, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:08:57.553 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 413696, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 101, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 505, 2026-02-13T19:08:57.554 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 101, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 101, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 101, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 404, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:08:57.555 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:08:57.556 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.557 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:08:57.558 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 101 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:08:57.559 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "1(1)", 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s1", 2026-02-13T19:08:57.560 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "1", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'101", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'101", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "0'0", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 101, 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.561 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 51, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 51, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 51, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.562 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'101", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 131, 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 45, 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:08:36.440408+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.563 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:08:36.859594+0000", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:08:36.859538+0000", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 51, 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "0'0", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "0'0", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.564 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 0, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:08:57.565 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 413696, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 101, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 505, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 1, 2026-02-13T19:08:57.566 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 101, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 101, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 101, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 404, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:08:57.567 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:08:57.568 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.569 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.570 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 101 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:08:57.571 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "2(4)", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s4", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "4", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'101", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'75", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "0'0", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 101, 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:08:57.572 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 51, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 51, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 51, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.573 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'101", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 137, 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 49, 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:08:40.834037+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:08:40.733179+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:08:40.733179+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:08:40.628858+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:08:36.440408+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:08:40.834037+0000", 2026-02-13T19:08:57.574 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:08:40.834037+0000", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:08:40.834037+0000", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 51, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "0'0", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "0'0", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 0, 2026-02-13T19:08:57.575 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 413696, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 101, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:08:57.576 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 505, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 1, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 101, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 101, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 404, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:08:57.577 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:08:57.578 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [], 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [], 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:08:57.579 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "peer": "4(2)", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s2", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "shard": "2", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "last_update": "45'101", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "last_complete": "45'101", 2026-02-13T19:08:57.580 INFO:tasks.workunit.client.0.trial034.stdout: "log_tail": "0'0", 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_user_version": 101, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill": "MAX", 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete": [], 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [], 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "history": { 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_created": 39, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "epoch_pool_created": 39, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_started": 51, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_interval_clean": 39, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_split": 0, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_marked_full": 0, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "same_up_since": 51, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "same_interval_since": 51, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "same_primary_since": 39, 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.581 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "prior_readable_until_ub": 0 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "stats": { 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "version": "45'101", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "reported_seq": 131, 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "reported_epoch": 45, 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "state": "active+remapped", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_fresh": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_change": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_active": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_peered": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean": "2026-02-13T19:08:36.440408+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_active": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_became_peered": "2026-02-13T19:08:36.860729+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_unstale": "2026-02-13T19:08:39.348016+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_undegraded": "2026-02-13T19:08:36.859594+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "last_fullsized": "2026-02-13T19:08:36.859538+0000", 2026-02-13T19:08:57.582 INFO:tasks.workunit.client.0.trial034.stdout: "mapping_epoch": 51, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "log_start": "0'0", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_start": "0'0", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "created": 39, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_clean": 40, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "parent": "0.0", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "parent_split_bits": 0, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub": "0'0", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub": "0'0", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_deep_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "last_clean_scrub_stamp": "2026-02-13T19:08:31.260267+0000", 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "objects_scrubbed": 0, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "log_size": 101, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "log_dups_size": 0, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "ondisk_log_size": 101, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "stats_invalid": false, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "dirty_stats_invalid": false, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "omap_stats_invalid": false, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_stats_invalid": false, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "hitset_bytes_stats_invalid": false, 2026-02-13T19:08:57.583 INFO:tasks.workunit.client.0.trial034.stdout: "pin_stats_invalid": false, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "manifest_stats_invalid": false, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrimq_len": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "last_scrub_duration": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "scrub_duration": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "objects_trimmed": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "snaptrim_duration": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "stat_sum": { 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes": 413696, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects": 101, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_clones": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_object_copies": 505, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing_on_primary": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_missing": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_degraded": 101, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_misplaced": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_unfound": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_dirty": 101, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_whiteouts": 0, 2026-02-13T19:08:57.584 INFO:tasks.workunit.client.0.trial034.stdout: "num_read": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_read_kb": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_write": 101, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_write_kb": 404, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_scrub_errors": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_shallow_scrub_errors": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_deep_scrub_errors": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_recovered": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_recovered": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_keys_recovered": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_omap": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_hit_set_archive": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_bytes_hit_set_archive": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_kb": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_kb": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_promote": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_high": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_flush_mode_low": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_some": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_evict_mode_full": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_pinned": 0, 2026-02-13T19:08:57.585 INFO:tasks.workunit.client.0.trial034.stdout: "num_legacy_snapsets": 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "num_large_omap_objects": 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_manifest": 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_bytes": 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "num_omap_keys": 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "num_objects_repaired": 0 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "up": [ 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "acting": [ 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 3, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 1, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 4, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 0, 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: 2 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "avail_no_missing": [ 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "3(0)", 2026-02-13T19:08:57.586 INFO:tasks.workunit.client.0.trial034.stdout: "0(3)", 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "1(1)", 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "4(2)" 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "object_location_counts": [ 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "objects": 101 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "blocked_by": [], 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "up_primary": 3, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "acting_primary": 3, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "purged_snaps": [] 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "empty": 0, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "dne": 0, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "incomplete": 0, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "last_epoch_started": 52, 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "hit_set_history": { 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "current_last_update": "0'0", 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: "history": [] 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.587 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_state": [ 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "name": "Started/Primary/Active", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "enter_time": "2026-02-13T19:08:41.982305+0000", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "might_have_unfound": [ 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "0(3)", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "1(1)", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "2(4)", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "osd": "4(2)", 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "status": "already probed" 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.588 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_progress": { 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "backfill_targets": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "waiting_on_backfill": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "last_backfill_started": "MIN", 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "backfill_info": { 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "begin": "MIN", 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "end": "MIN", 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "objects": [] 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "peer_backfill_info": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "backfills_in_flight": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "recovering": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "pg_backend": { 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "recovery_ops": [], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "read_ops": [] 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: { 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "name": "Started", 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "enter_time": "2026-02-13T19:08:41.882797+0000" 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: ], 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "scrubber": { 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "active": false, 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "must_scrub": false, 2026-02-13T19:08:57.589 INFO:tasks.workunit.client.0.trial034.stdout: "must_deep_scrub": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "must_repair": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "schedule": "not registered for scrubbing", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "shallow-target": { 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "level": "shallow", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "orig_sched_time": "2026-02-14T23:31:32.013403+0000", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "last_issue": "ok", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "forced": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "eligible": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "queued": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "active": false 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "deep-target": { 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "pgid": "2.0s0", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "level": "deep", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "urgency": "periodic-regular", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "sched_time": "2026-02-21T19:36:09.800836+0000", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "orig_sched_time": "2026-02-21T19:36:09.800836+0000", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "last_issue": "ok", 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "forced": false, 2026-02-13T19:08:57.590 INFO:tasks.workunit.client.0.trial034.stdout: "eligible": false, 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout: "queued": false, 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout: "active": false 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout: } 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout: }, 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout: "agent_state": {} 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stdout:} 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:08:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-13T19:08:57.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:08:57.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-13T19:08:57.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-13T19:08:57.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-13T19:08:57.670 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-13T19:08:57.670 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-13T19:08:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-13T19:08:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-13T19:08:57.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-13T19:09:02.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-13T19:09:02.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-13T19:09:02.803 INFO:tasks.workunit.client.0.trial034.stderr:pg has no unfound objects 2026-02-13T19:09:02.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-13T19:09:02.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:09:02.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:09:02.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:09:02.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:09:02.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:09:02.809 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:09:02.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:09:02.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:09:02.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:09:02.897 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:4' 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:03.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:09:03.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803789 2026-02-13T19:09:03.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803789 2026-02-13T19:09:03.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789' 2026-02-13T19:09:03.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:03.188 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:09:03.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607566 2026-02-13T19:09:03.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607566 2026-02-13T19:09:03.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566' 2026-02-13T19:09:03.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:03.253 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:09:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-13T19:09:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-13T19:09:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-201863462919' 2026-02-13T19:09:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:03.317 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:09:03.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182413 2026-02-13T19:09:03.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182413 2026-02-13T19:09:03.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-201863462919 3-107374182413' 2026-02-13T19:09:03.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:03.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:09:03.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888076 2026-02-13T19:09:03.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888076 2026-02-13T19:09:03.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-51539607566 2-201863462919 3-107374182413 4-146028888076' 2026-02-13T19:09:03.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:03.445 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803789 2026-02-13T19:09:03.445 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:03.446 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:09:03.446 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803789 2026-02-13T19:09:03.447 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:03.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803789 2026-02-13T19:09:03.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-13T19:09:03.448 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803789 2026-02-13T19:09:03.448 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:03.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-13T19:09:03.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:09:04.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:09:04.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:04.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-13T19:09:04.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:04.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607566 2026-02-13T19:09:04.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:04.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:09:04.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607566 2026-02-13T19:09:04.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:04.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607566 2026-02-13T19:09:04.909 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607566 2026-02-13T19:09:04.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607566' 2026-02-13T19:09:04.910 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:09:05.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607566 -lt 51539607566 2026-02-13T19:09:05.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:05.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-201863462919 2026-02-13T19:09:05.138 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:05.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:09:05.139 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-201863462919 2026-02-13T19:09:05.139 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:05.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462919 2026-02-13T19:09:05.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 201863462919' 2026-02-13T19:09:05.140 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 201863462919 2026-02-13T19:09:05.140 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:09:05.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-13T19:09:05.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:05.370 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182413 2026-02-13T19:09:05.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:05.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:09:05.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182413 2026-02-13T19:09:05.372 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:05.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182413 2026-02-13T19:09:05.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182413' 2026-02-13T19:09:05.373 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182413 2026-02-13T19:09:05.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:09:05.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182413 -lt 107374182413 2026-02-13T19:09:05.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:05.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888076 2026-02-13T19:09:05.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:05.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:09:05.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888076 2026-02-13T19:09:05.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:05.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888076 2026-02-13T19:09:05.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888076' 2026-02-13T19:09:05.604 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888076 2026-02-13T19:09:05.605 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:09:05.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888076 -lt 146028888076 2026-02-13T19:09:05.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:09:05.830 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:05.830 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:06.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:09:06.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:06.141 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:06.141 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:06.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:06.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:06.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:06.142 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:06.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:09:06.360 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:06.361 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:06.361 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:06.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:09:06.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:09:06.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:09:06.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-13T19:09:06.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-13T19:09:06.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-13T19:09:06.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-13T19:09:06.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-13T19:09:06.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-13T19:09:06.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-13T19:09:06.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-13T19:09:06.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-13T19:09:06.794 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-13T19:09:06.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-13T19:09:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-13T19:09:06.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-13T19:09:06.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-13T19:09:06.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-13T19:09:06.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-13T19:09:06.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-13T19:09:06.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-13T19:09:06.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-13T19:09:06.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-13T19:09:06.975 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-13T19:09:06.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:06.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:06.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-13T19:09:06.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-13T19:09:07.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.025 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-13T19:09:07.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-13T19:09:07.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-13T19:09:07.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-13T19:09:07.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-13T19:09:07.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-13T19:09:07.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-13T19:09:07.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-13T19:09:07.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-13T19:09:07.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-13T19:09:07.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-13T19:09:07.194 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-13T19:09:07.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-13T19:09:07.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-13T19:09:07.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-13T19:09:07.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.279 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.280 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-13T19:09:07.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-13T19:09:07.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-13T19:09:07.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-13T19:09:07.332 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-13T19:09:07.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-13T19:09:07.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-13T19:09:07.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-13T19:09:07.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-13T19:09:07.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-13T19:09:07.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-13T19:09:07.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-13T19:09:07.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-13T19:09:07.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-13T19:09:07.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-13T19:09:07.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-13T19:09:07.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-13T19:09:07.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.568 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-13T19:09:07.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.585 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-13T19:09:07.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-13T19:09:07.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-13T19:09:07.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-13T19:09:07.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-13T19:09:07.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-13T19:09:07.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-13T19:09:07.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.702 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.702 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-13T19:09:07.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.718 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.719 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.719 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-13T19:09:07.720 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-13T19:09:07.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-13T19:09:07.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-13T19:09:07.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-13T19:09:07.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-13T19:09:07.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-13T19:09:07.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-13T19:09:07.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.850 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-13T19:09:07.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-13T19:09:07.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-13T19:09:07.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-13T19:09:07.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-13T19:09:07.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.932 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-13T19:09:07.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-13T19:09:07.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-13T19:09:07.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-13T19:09:07.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-13T19:09:07.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:07.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:07.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-13T19:09:07.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-13T19:09:08.002 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-13T19:09:08.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-13T19:09:08.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-13T19:09:08.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-13T19:09:08.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-13T19:09:08.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.101 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-13T19:09:08.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-13T19:09:08.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-13T19:09:08.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-13T19:09:08.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-13T19:09:08.171 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-13T19:09:08.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-13T19:09:08.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-13T19:09:08.221 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-13T19:09:08.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.253 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-13T19:09:08.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-13T19:09:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-13T19:09:08.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-13T19:09:08.304 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-13T19:09:08.321 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-13T19:09:08.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-13T19:09:08.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-13T19:09:08.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.354 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-13T19:09:08.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:09:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:09:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:09:08.630 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:09:08.638 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:09:08.920 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:09:08.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:09:08.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:09:08.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:09:08.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:09:08.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:08.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:08.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:08.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:08.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:09.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:09.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:09:09.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:09:09.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:09:09.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:09:09.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:09:09.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:09:09.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:09.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:09:09.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:09:09.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:09.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:09:09.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:09:09.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:09:09.065 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:09:09.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.066 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:09:09.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:09.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:09.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:09.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:09.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:09:09.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:09:09.070 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:09:09.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:09:09.071 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:09:09.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:09:09.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:09.072 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:09:09.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:09:09.073 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:09.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:09:09.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:09:09.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:09:09.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:09:09.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:09:09.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:09:09.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:09:09.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:09:09.078 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:09:09.078 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.078 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:09:09.079 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:09:09.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:09.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:09:09.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:09:09.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:09.124 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:09:09.125 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:09:09.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:09:09.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:09:09.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:09:09.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:09:09.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.170 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:09.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:09:09.171 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:09:09.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:09:09.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:09:09.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:09:09.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:09:09.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:09:09.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:09:09.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:09:09.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:09:09.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:09.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:09.313 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:09.313 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:09.313 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:09.313 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:09.313 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:09:09.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:09:09.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:09:09.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:09:09.422 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:09:09.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:10.432 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:10.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:10.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:09:10.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:09:10.471 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:09:10.475 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:09:10.476 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:09:09.115+0000 7fadbb04dc00 0 load: jerasure load: lrc 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:09:10.477 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-13T19:09:10.478 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-13T19:09:10.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:10.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:10.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:09:10.484 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:10.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=162b0ba7-e32e-4a52-8efd-88ef9d4e863b 2026-02-13T19:09:10.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 162b0ba7-e32e-4a52-8efd-88ef9d4e863b' 2026-02-13T19:09:10.485 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 162b0ba7-e32e-4a52-8efd-88ef9d4e863b 2026-02-13T19:09:10.485 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:10.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDWdo9pbQFIHRAAu8fgnzVBViAKFzoIlNdX8Q== 2026-02-13T19:09:10.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDWdo9pbQFIHRAAu8fgnzVBViAKFzoIlNdX8Q=="}' 2026-02-13T19:09:10.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 162b0ba7-e32e-4a52-8efd-88ef9d4e863b -i td/test-erasure-eio/0/new.json 2026-02-13T19:09:10.594 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:10.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:09:10.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWdo9pbQFIHRAAu8fgnzVBViAKFzoIlNdX8Q== --osd-uuid 162b0ba7-e32e-4a52-8efd-88ef9d4e863b 2026-02-13T19:09:10.618 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:10.617+0000 7fdff2e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:10.620 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:10.619+0000 7fdff2e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:10.620 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:10.620+0000 7fdff2e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:10.621 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:10.620+0000 7fdff2e3d780 -1 bdev(0x42dcc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:10.621 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:10.620+0000 7fdff2e3d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:09:11.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:09:11.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:11.030 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:09:11.031 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:09:11.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:11.133 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:09:11.134 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:09:11.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:11.134 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:11.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:11.135 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:11.149 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:11.148+0000 7f0e3d459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:11.151 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:11.150+0000 7f0e3d459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:11.152 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:11.151+0000 7f0e3d459780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:11.238 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:11.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:11.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:09:11.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:11.375 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:11.374+0000 7f0e3d459780 -1 Falling back to public interface 2026-02-13T19:09:11.607 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:11.606+0000 7f0e3d459780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:09:12.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:12.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:12.338 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:12.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:12.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:12.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:09:12.565 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3135491866,v1:127.0.0.1:6803/3135491866] [v2:127.0.0.1:6804/3135491866,v1:127.0.0.1:6805/3135491866] exists,up 162b0ba7-e32e-4a52-8efd-88ef9d4e863b 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:12.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:12.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:09:12.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:12.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b4612937-a988-499d-b3ed-df350079a9d8 2026-02-13T19:09:12.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b4612937-a988-499d-b3ed-df350079a9d8' 2026-02-13T19:09:12.569 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 b4612937-a988-499d-b3ed-df350079a9d8 2026-02-13T19:09:12.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYdo9pwOKIIhAAYmUImMSUUq7t1lOp42gUEA== 2026-02-13T19:09:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYdo9pwOKIIhAAYmUImMSUUq7t1lOp42gUEA=="}' 2026-02-13T19:09:12.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b4612937-a988-499d-b3ed-df350079a9d8 -i td/test-erasure-eio/1/new.json 2026-02-13T19:09:12.809 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:12.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:09:12.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDYdo9pwOKIIhAAYmUImMSUUq7t1lOp42gUEA== --osd-uuid b4612937-a988-499d-b3ed-df350079a9d8 2026-02-13T19:09:12.834 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:12.833+0000 7f1d3f641780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:12.835 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:12.834+0000 7f1d3f641780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:12.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:12.835+0000 7f1d3f641780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:12.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:12.836+0000 7f1d3f641780 -1 bdev(0x2b36c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:12.837 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:12.836+0000 7f1d3f641780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:09:13.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:09:13.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:13.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:09:13.165 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:09:13.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:13.481 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:09:13.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:09:13.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:13.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:13.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:13.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:13.496 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:13.495+0000 7f1c34635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:13.498 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:13.497+0000 7f1c34635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:13.498 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:13.498+0000 7f1c34635780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:13.628 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:13.627+0000 7f1c34635780 -1 Falling back to public interface 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:13.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:13.707 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:13.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:13.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:09:13.913 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:13.912+0000 7f1c34635780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:09:13.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:14.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:14.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:14.933 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:14.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:14.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:14.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:09:15.161 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3049158144,v1:127.0.0.1:6811/3049158144] [v2:127.0.0.1:6812/3049158144,v1:127.0.0.1:6813/3049158144] exists,up b4612937-a988-499d-b3ed-df350079a9d8 2026-02-13T19:09:15.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:15.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:15.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:15.167 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:15.168 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=43e875d6-ea39-48b7-9c34-c7bf286f006a 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 43e875d6-ea39-48b7-9c34-c7bf286f006a' 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 43e875d6-ea39-48b7-9c34-c7bf286f006a 2026-02-13T19:09:15.169 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:15.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDbdo9pgA5rChAAmmiIcf1gkddK1vf+YmZPwA== 2026-02-13T19:09:15.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDbdo9pgA5rChAAmmiIcf1gkddK1vf+YmZPwA=="}' 2026-02-13T19:09:15.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 43e875d6-ea39-48b7-9c34-c7bf286f006a -i td/test-erasure-eio/2/new.json 2026-02-13T19:09:15.404 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:09:15.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:09:15.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDbdo9pgA5rChAAmmiIcf1gkddK1vf+YmZPwA== --osd-uuid 43e875d6-ea39-48b7-9c34-c7bf286f006a 2026-02-13T19:09:15.429 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:15.428+0000 7f288da71780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:15.430 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:15.430+0000 7f288da71780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:15.431 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:15.430+0000 7f288da71780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:15.432 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:15.431+0000 7f288da71780 -1 bdev(0x40acc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:15.432 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:15.431+0000 7f288da71780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:09:15.765 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:09:15.765 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:15.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:09:15.766 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:09:15.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:16.082 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:09:16.083 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:09:16.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:16.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:16.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:16.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:16.098 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:16.097+0000 7f7ed1017780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:16.100 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:16.099+0000 7f7ed1017780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:16.100 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:16.100+0000 7f7ed1017780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:16.258 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:16.258+0000 7f7ed1017780 -1 Falling back to public interface 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:16.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:16.309 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:16.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:16.309 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:09:16.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:16.619 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:16.618+0000 7f7ed1017780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:09:17.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:17.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:17.536 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:17.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:17.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:17.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:09:17.764 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2076415998,v1:127.0.0.1:6819/2076415998] [v2:127.0.0.1:6820/2076415998,v1:127.0.0.1:6821/2076415998] exists,up 43e875d6-ea39-48b7-9c34-c7bf286f006a 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:17.768 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:17.769 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:17.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=da600ea7-8ae3-4660-849a-b7a7fb195fa5 2026-02-13T19:09:17.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 da600ea7-8ae3-4660-849a-b7a7fb195fa5' 2026-02-13T19:09:17.772 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 da600ea7-8ae3-4660-849a-b7a7fb195fa5 2026-02-13T19:09:17.772 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:17.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDddo9prV5RLhAAa+zHs4iJCnORInONn5JdPg== 2026-02-13T19:09:17.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDddo9prV5RLhAAa+zHs4iJCnORInONn5JdPg=="}' 2026-02-13T19:09:17.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new da600ea7-8ae3-4660-849a-b7a7fb195fa5 -i td/test-erasure-eio/3/new.json 2026-02-13T19:09:18.007 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:09:18.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:09:18.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDddo9prV5RLhAAa+zHs4iJCnORInONn5JdPg== --osd-uuid da600ea7-8ae3-4660-849a-b7a7fb195fa5 2026-02-13T19:09:18.032 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.032+0000 7f953161a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.034 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.033+0000 7f953161a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.034+0000 7f953161a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.034+0000 7f953161a780 -1 bdev(0x2e2ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:18.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.034+0000 7f953161a780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:09:18.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:09:18.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:18.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:09:18.364 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:09:18.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:18.677 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:09:18.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:09:18.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:18.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:18.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:18.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:18.691 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.690+0000 7f495fe41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.693 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.692+0000 7f495fe41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.694 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.693+0000 7f495fe41780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:18.823 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:18.822+0000 7f495fe41780 -1 Falling back to public interface 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:18.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:18.909 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:18.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:18.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:18.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:09:19.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:19.172 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:19.171+0000 7f495fe41780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:20.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:09:20.367 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1417265406,v1:127.0.0.1:6827/1417265406] [v2:127.0.0.1:6828/1417265406,v1:127.0.0.1:6829/1417265406] exists,up da600ea7-8ae3-4660-849a-b7a7fb195fa5 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:20.368 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:20.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-13T19:09:20.370 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:20.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=12e32c89-e44b-467d-baed-93d79ef24f45 2026-02-13T19:09:20.371 INFO:tasks.workunit.client.0.trial034.stdout:add osd4 12e32c89-e44b-467d-baed-93d79ef24f45 2026-02-13T19:09:20.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 12e32c89-e44b-467d-baed-93d79ef24f45' 2026-02-13T19:09:20.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:20.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgdo9pIn6rFhAAOr3PdMGk2efz4js/4vuNtg== 2026-02-13T19:09:20.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgdo9pIn6rFhAAOr3PdMGk2efz4js/4vuNtg=="}' 2026-02-13T19:09:20.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 12e32c89-e44b-467d-baed-93d79ef24f45 -i td/test-erasure-eio/4/new.json 2026-02-13T19:09:20.611 INFO:tasks.workunit.client.0.trial034.stdout:4 2026-02-13T19:09:20.619 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-13T19:09:20.620 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgdo9pIn6rFhAAOr3PdMGk2efz4js/4vuNtg== --osd-uuid 12e32c89-e44b-467d-baed-93d79ef24f45 2026-02-13T19:09:20.636 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:20.636+0000 7f78a2071780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:20.638 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:20.637+0000 7f78a2071780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:20.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:20.638+0000 7f78a2071780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:20.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:20.638+0000 7f78a2071780 -1 bdev(0x2ca6c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:20.639 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:20.638+0000 7f78a2071780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-13T19:09:20.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-13T19:09:20.967 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:20.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-13T19:09:20.968 INFO:tasks.workunit.client.0.trial034.stdout:adding osd4 key to auth repository 2026-02-13T19:09:20.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:21.294 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-13T19:09:21.294 INFO:tasks.workunit.client.0.trial034.stdout:start osd.4 2026-02-13T19:09:21.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:21.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:21.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:21.295 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:21.309 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:21.309+0000 7f4be4856780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:21.311 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:21.310+0000 7f4be4856780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:21.312 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:21.311+0000 7f4be4856780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:21.442 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:21.441+0000 7f4be4856780 -1 Falling back to public interface 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:21.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:21.521 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:21.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:21.521 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:09:21.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:21.795 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:21.794+0000 7f4be4856780 -1 osd.4 0 log_to_monitors true 2026-02-13T19:09:22.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:22.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:22.749 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:22.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:22.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:22.749 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-13T19:09:22.975 INFO:tasks.workunit.client.0.trial034.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1972074198,v1:127.0.0.1:6835/1972074198] [v2:127.0.0.1:6836/1972074198,v1:127.0.0.1:6837/1972074198] exists,up 12e32c89-e44b-467d-baed-93d79ef24f45 2026-02-13T19:09:22.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:22.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:22.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:22.981 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:22.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:22.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:22.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-13T19:09:22.983 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:22.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6b7a707c-2665-4c01-aa48-48b613e22a2f 2026-02-13T19:09:22.983 INFO:tasks.workunit.client.0.trial034.stdout:add osd5 6b7a707c-2665-4c01-aa48-48b613e22a2f 2026-02-13T19:09:22.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 6b7a707c-2665-4c01-aa48-48b613e22a2f' 2026-02-13T19:09:22.984 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:22.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDido9pWR/xOhAAxxqxmn0F+yJc+9inaEqxZQ== 2026-02-13T19:09:22.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDido9pWR/xOhAAxxqxmn0F+yJc+9inaEqxZQ=="}' 2026-02-13T19:09:22.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6b7a707c-2665-4c01-aa48-48b613e22a2f -i td/test-erasure-eio/5/new.json 2026-02-13T19:09:23.218 INFO:tasks.workunit.client.0.trial034.stdout:5 2026-02-13T19:09:23.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-13T19:09:23.227 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDido9pWR/xOhAAxxqxmn0F+yJc+9inaEqxZQ== --osd-uuid 6b7a707c-2665-4c01-aa48-48b613e22a2f 2026-02-13T19:09:23.243 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.242+0000 7f9308843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:23.244 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.243+0000 7f9308843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:23.245 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.244+0000 7f9308843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:23.245 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.244+0000 7f9308843780 -1 bdev(0x2cf2c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:23.245 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.244+0000 7f9308843780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-13T19:09:23.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-13T19:09:23.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:23.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-13T19:09:23.566 INFO:tasks.workunit.client.0.trial034.stdout:adding osd5 key to auth repository 2026-02-13T19:09:23.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:23.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-13T19:09:23.890 INFO:tasks.workunit.client.0.trial034.stdout:start osd.5 2026-02-13T19:09:23.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:23.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:23.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:23.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:23.905 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.904+0000 7f64a7241780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:23.906 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.906+0000 7f64a7241780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:23.907 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:23.906+0000 7f64a7241780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:24.044 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:24.043+0000 7f64a7241780 -1 Falling back to public interface 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:24.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:24.117 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:24.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:24.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:09:24.316 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:24.315+0000 7f64a7241780 -1 osd.5 0 log_to_monitors true 2026-02-13T19:09:24.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:25.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:25.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:25.344 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:25.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:25.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:25.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-13T19:09:25.566 INFO:tasks.workunit.client.0.trial034.stdout:osd.5 up in weight 1 up_from 43 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/4043279605,v1:127.0.0.1:6843/4043279605] [v2:127.0.0.1:6844/4043279605,v1:127.0.0.1:6845/4043279605] exists,up 6b7a707c-2665-4c01-aa48-48b613e22a2f 2026-02-13T19:09:25.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-13T19:09:25.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:25.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:25.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5ec3bc7a-2341-4787-bb8d-cd6713812818 2026-02-13T19:09:25.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 5ec3bc7a-2341-4787-bb8d-cd6713812818' 2026-02-13T19:09:25.576 INFO:tasks.workunit.client.0.trial034.stdout:add osd6 5ec3bc7a-2341-4787-bb8d-cd6713812818 2026-02-13T19:09:25.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:25.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDldo9pz76MIhAAt+JG9BL7Q28RNbBI7rja2w== 2026-02-13T19:09:25.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDldo9pz76MIhAAt+JG9BL7Q28RNbBI7rja2w=="}' 2026-02-13T19:09:25.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5ec3bc7a-2341-4787-bb8d-cd6713812818 -i td/test-erasure-eio/6/new.json 2026-02-13T19:09:25.810 INFO:tasks.workunit.client.0.trial034.stdout:6 2026-02-13T19:09:25.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-13T19:09:25.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDldo9pz76MIhAAt+JG9BL7Q28RNbBI7rja2w== --osd-uuid 5ec3bc7a-2341-4787-bb8d-cd6713812818 2026-02-13T19:09:25.833 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:25.833+0000 7f5ac2262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:25.835 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:25.834+0000 7f5ac2262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:25.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:25.835+0000 7f5ac2262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:25.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:25.835+0000 7f5ac2262780 -1 bdev(0x39bec00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:25.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:25.835+0000 7f5ac2262780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-13T19:09:26.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-13T19:09:26.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:26.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-13T19:09:26.166 INFO:tasks.workunit.client.0.trial034.stdout:adding osd6 key to auth repository 2026-02-13T19:09:26.167 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:26.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-13T19:09:26.486 INFO:tasks.workunit.client.0.trial034.stdout:start osd.6 2026-02-13T19:09:26.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:26.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:26.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:26.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:26.501 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:26.500+0000 7f87ecc14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:26.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:26.502+0000 7f87ecc14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:26.503 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:26.503+0000 7f87ecc14780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:26.655 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:26.654+0000 7f87ecc14780 -1 Falling back to public interface 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:26.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:26.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:26.714 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:26.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:26.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:09:26.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:26.970 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:26.969+0000 7f87ecc14780 -1 osd.6 0 log_to_monitors true 2026-02-13T19:09:27.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:27.944 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:27.944 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:27.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:27.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:27.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-13T19:09:28.172 INFO:tasks.workunit.client.0.trial034.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2594400226,v1:127.0.0.1:6851/2594400226] [v2:127.0.0.1:6852/2594400226,v1:127.0.0.1:6853/2594400226] exists,up 5ec3bc7a-2341-4787-bb8d-cd6713812818 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:09:28.174 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:28.175 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:28.175 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:28.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:09:28.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:09:28.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:09:28.218 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:09:28.222 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:09:28.224 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:09:11.377+0000 7f0e3d459780 0 load: jerasure load: lrc 2026-02-13T19:09:28.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:09:28.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-13T19:09:28.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:09:28.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:09:28.908 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:09:28.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:09:29.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:09:29.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:09:29.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:09:29.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:09:29.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:09:30.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:09:30.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.231 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:09:30.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-13T19:09:30.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-13T19:09:30.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-13T19:09:30.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:09:30.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-13T19:09:30.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-13T19:09:30.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-13T19:09:30.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.365 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:09:30.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-13T19:09:30.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-13T19:09:30.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-13T19:09:30.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.431 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:09:30.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-13T19:09:30.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-13T19:09:30.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182406' 2026-02-13T19:09:30.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.494 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:09:30.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-13T19:09:30.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-13T19:09:30.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182406 4-146028888068' 2026-02-13T19:09:30.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:09:30.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-13T19:09:30.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-13T19:09:30.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182406 4-146028888068 5-184683593732' 2026-02-13T19:09:30.629 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:30.629 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:09:30.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299396 2026-02-13T19:09:30.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299396 2026-02-13T19:09:30.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-107374182406 4-146028888068 5-184683593732 6-223338299396' 2026-02-13T19:09:30.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:30.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-13T19:09:30.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:30.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:09:30.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-13T19:09:30.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:30.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-13T19:09:30.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-13T19:09:30.696 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803782 2026-02-13T19:09:30.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:30.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-13T19:09:30.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:09:31.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:09:31.920 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:32.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-13T19:09:32.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:32.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-13T19:09:32.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:32.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:09:32.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-13T19:09:32.150 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:32.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-13T19:09:32.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-13T19:09:32.151 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607558 2026-02-13T19:09:32.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:09:32.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-13T19:09:32.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:32.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-13T19:09:32.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:32.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:09:32.378 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-13T19:09:32.378 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:32.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-13T19:09:32.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-13T19:09:32.379 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411333 2026-02-13T19:09:32.379 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:09:32.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-13T19:09:32.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:32.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182406 2026-02-13T19:09:32.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:32.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:09:32.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182406 2026-02-13T19:09:32.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:32.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182406 2026-02-13T19:09:32.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-13T19:09:32.606 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182406 2026-02-13T19:09:32.606 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:09:32.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-13T19:09:32.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:32.831 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888068 2026-02-13T19:09:32.831 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:32.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:09:32.832 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-13T19:09:32.832 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:32.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888068 2026-02-13T19:09:32.833 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-13T19:09:32.834 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888068 2026-02-13T19:09:32.834 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:09:33.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-13T19:09:33.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593732 2026-02-13T19:09:33.061 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:33.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:09:33.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-13T19:09:33.062 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:33.063 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-13T19:09:33.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-13T19:09:33.064 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593732 2026-02-13T19:09:33.064 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:09:33.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-13T19:09:33.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:33.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-223338299396 2026-02-13T19:09:33.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:33.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-13T19:09:33.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299396 2026-02-13T19:09:33.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:33.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299396 2026-02-13T19:09:33.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 223338299396' 2026-02-13T19:09:33.292 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.6 seq 223338299396 2026-02-13T19:09:33.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-13T19:09:33.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299397 -lt 223338299396 2026-02-13T19:09:33.516 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:09:33.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:33.517 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:33.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:09:33.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:33.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:34.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:09:34.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:34.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:34.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:34.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:09:34.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:09:34.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:34.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:09:34.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-13T19:09:34.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-13T19:09:34.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:09:34.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-13T19:09:34.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:09:34.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:09:34.385 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:09:34.385 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-13T19:09:34.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:09:34.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:09:34.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:09:34.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:09:34.618 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:09:34.619 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:09:34.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:09:34.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:09:34.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:34.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:09:34.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:09:34.668 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-13T19:09:34.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-13T19:09:34.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:09:34.717 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-13T19:09:34.717 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-13T19:09:34.717 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:09:34.950 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:09:34.950 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:09:34.950 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-13T19:09:34.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-13T19:09:34.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:34.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:34.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:34.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:34.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:35.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:35.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-13T19:09:35.485 INFO:tasks.workunit.client.0.trial034.stderr:osd.6 is already down. 2026-02-13T19:09:35.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-13T19:09:35.752 INFO:tasks.workunit.client.0.trial034.stderr:osd.6 is already out. 2026-02-13T19:09:35.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-13T19:09:35.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:09:35.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:09:35.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:09:35.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:09:35.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:09:35.761 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:09:35.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:09:35.761 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:09:35.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:09:35.851 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:3 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:4 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:5 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:6' 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.079 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:09:36.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-13T19:09:36.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-13T19:09:36.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-13T19:09:36.146 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:09:36.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-13T19:09:36.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-13T19:09:36.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561' 2026-02-13T19:09:36.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.211 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:09:36.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-13T19:09:36.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-13T19:09:36.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336' 2026-02-13T19:09:36.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.275 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:09:36.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182409 2026-02-13T19:09:36.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182409 2026-02-13T19:09:36.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182409' 2026-02-13T19:09:36.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-13T19:09:36.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-13T19:09:36.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-13T19:09:36.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182409 4-146028888071' 2026-02-13T19:09:36.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.403 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-13T19:09:36.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-13T19:09:36.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-13T19:09:36.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-107374182409 4-146028888071 5-184683593735' 2026-02-13T19:09:36.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:36.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-13T19:09:36.513 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:36.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:09:36.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-13T19:09:36.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:36.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-13T19:09:36.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-13T19:09:36.516 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803785 2026-02-13T19:09:36.516 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:36.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-13T19:09:36.747 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:09:37.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:09:37.749 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:37.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-13T19:09:37.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:37.975 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-13T19:09:37.975 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:37.976 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:09:37.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-13T19:09:37.976 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:37.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-13T19:09:37.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-13T19:09:37.978 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607561 2026-02-13T19:09:37.978 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:09:38.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-13T19:09:38.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:38.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-13T19:09:38.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:38.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:09:38.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-13T19:09:38.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:38.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-13T19:09:38.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-13T19:09:38.208 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411336 2026-02-13T19:09:38.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:09:38.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-13T19:09:38.436 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:38.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182409 2026-02-13T19:09:38.437 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:38.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:09:38.439 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182409 2026-02-13T19:09:38.439 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:38.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182409 2026-02-13T19:09:38.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-13T19:09:38.440 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182409 2026-02-13T19:09:38.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:09:38.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-13T19:09:38.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:38.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888071 2026-02-13T19:09:38.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:38.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-13T19:09:38.672 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888071 2026-02-13T19:09:38.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888071 2026-02-13T19:09:38.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-13T19:09:38.674 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.4 seq 146028888071 2026-02-13T19:09:38.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-13T19:09:38.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-13T19:09:38.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:38.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-13T19:09:38.906 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:38.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-13T19:09:38.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-13T19:09:38.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:38.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-13T19:09:38.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-13T19:09:38.909 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.5 seq 184683593735 2026-02-13T19:09:38.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-13T19:09:39.131 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-13T19:09:39.132 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:09:39.132 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:39.132 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:39.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:09:39.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:39.454 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:39.454 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:39.454 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:39.454 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:39.455 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:39.455 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:39.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:09:39.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:39.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:39.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T19:09:39.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-13T19:09:40.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T19:09:40.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:40.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:40.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:40.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:40.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:40.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:40.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:40.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-13T19:09:40.323 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:40.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:40.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:40.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-13T19:09:40.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-13T19:09:40.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-13T19:09:40.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-13T19:09:40.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-13T19:09:40.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-13T19:09:40.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-13T19:09:40.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-13T19:09:40.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-13T19:09:41.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-13T19:09:41.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:41.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:41.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:09:41.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:41.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:41.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:09:41.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:09:41.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-13T19:09:41.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:09:41.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:09:41.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-13T19:09:41.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:09:41.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:09:41.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:09:41.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:09:41.713 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:09:41.988 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:09:41.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:09:42.276 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:42.284 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:42.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:42.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:42.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:42.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:42.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:09:42.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:09:42.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:09:42.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:09:42.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:09:42.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:09:42.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:42.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:09:42.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:09:42.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:42.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:09:42.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:09:42.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:09:42.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:09:42.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:09:42.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:42.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:42.419 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:42.419 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:42.419 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:42.419 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:09:42.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:09:42.420 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:09:42.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:09:42.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:09:42.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:09:42.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:42.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:09:42.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:09:42.423 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:09:42.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:09:42.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:09:42.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:09:42.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:09:42.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:09:42.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:09:42.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:09:42.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:09:42.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:09:42.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:09:42.430 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:09:42.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:09:42.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:09:42.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:09:42.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:42.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:09:42.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:09:42.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:09:42.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:09:42.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:09:42.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:09:42.474 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:09:42.475 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:09:42.476 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:09:42.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:09:42.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:09:42.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:09:42.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:09:42.520 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:09:42.522 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:09:42.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:09:42.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:09:42.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:09:42.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:09:42.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:09:42.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:09:42.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:09:42.665 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:42.666 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:09:42.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:09:42.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:09:42.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:09:42.777 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:09:42.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:09:43.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:09:43.827 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:09:43.832 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:09:43.833 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:09:42.466+0000 7fe93dc83c00 0 load: jerasure load: lrc 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:09:43.834 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:09:43.835 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:09:43.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:43.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:09:43.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:43.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:43.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:09:43.841 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:43.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=471ae875-a1aa-4a0e-983f-77f1d39d1d87 2026-02-13T19:09:43.842 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 471ae875-a1aa-4a0e-983f-77f1d39d1d87 2026-02-13T19:09:43.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 471ae875-a1aa-4a0e-983f-77f1d39d1d87' 2026-02-13T19:09:43.843 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:43.849 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3do9pgT6LMhAAexKtUMVWRP1xOfvHeNXjMA== 2026-02-13T19:09:43.849 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3do9pgT6LMhAAexKtUMVWRP1xOfvHeNXjMA=="}' 2026-02-13T19:09:43.849 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 471ae875-a1aa-4a0e-983f-77f1d39d1d87 -i td/test-erasure-eio/0/new.json 2026-02-13T19:09:43.951 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:43.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:09:43.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3do9pgT6LMhAAexKtUMVWRP1xOfvHeNXjMA== --osd-uuid 471ae875-a1aa-4a0e-983f-77f1d39d1d87 2026-02-13T19:09:43.974 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:43.973+0000 7f190f265780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:43.975 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:43.975+0000 7f190f265780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:43.976 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:43.975+0000 7f190f265780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:43.977 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:43.976+0000 7f190f265780 -1 bdev(0x3f68c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:43.977 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:43.976+0000 7f190f265780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:09:44.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:09:44.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:44.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:09:44.377 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:09:44.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:44.481 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:09:44.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:09:44.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:44.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:44.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:44.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:44.496 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:44.495+0000 7f3b659d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:44.497 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:44.496+0000 7f3b659d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:44.498 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:44.497+0000 7f3b659d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:44.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:44.581 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:44.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:44.582 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:09:44.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:44.719 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:44.718+0000 7f3b659d8780 -1 Falling back to public interface 2026-02-13T19:09:44.989 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:44.988+0000 7f3b659d8780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:09:45.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:45.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:45.681 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:45.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:45.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:45.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:09:45.906 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1095302607,v1:127.0.0.1:6803/1095302607] [v2:127.0.0.1:6804/1095302607,v1:127.0.0.1:6805/1095302607] exists,up 471ae875-a1aa-4a0e-983f-77f1d39d1d87 2026-02-13T19:09:45.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:45.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:45.912 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:45.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 fc346775-6cc9-48ef-82e2-6c58bbc7a88a' 2026-02-13T19:09:45.914 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:09:45.915 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:45.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD5do9pmhfTNhAAMZljT09hqS3eBklHrJH7oQ== 2026-02-13T19:09:45.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD5do9pmhfTNhAAMZljT09hqS3eBklHrJH7oQ=="}' 2026-02-13T19:09:45.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fc346775-6cc9-48ef-82e2-6c58bbc7a88a -i td/test-erasure-eio/1/new.json 2026-02-13T19:09:46.155 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:46.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:09:46.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5do9pmhfTNhAAMZljT09hqS3eBklHrJH7oQ== --osd-uuid fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:09:46.179 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.178+0000 7fc1fee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.180 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.180+0000 7fc1fee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.181 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.181+0000 7fc1fee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.182 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.181+0000 7fc1fee44780 -1 bdev(0x2896c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:46.182 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.181+0000 7fc1fee44780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:09:46.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:09:46.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:46.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:09:46.512 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:09:46.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:46.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:09:46.830 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:09:46.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:46.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:46.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:46.831 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:46.843 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.842+0000 7fd4d823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.845 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.844+0000 7fd4d823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.845 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.845+0000 7fd4d823b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:46.977 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:46.976+0000 7fd4d823b780 -1 Falling back to public interface 2026-02-13T19:09:47.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:09:47.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:47.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:09:47.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:47.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:47.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:47.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:47.054 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:47.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:47.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:09:47.282 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:47.280+0000 7fd4d823b780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:09:47.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:48.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:48.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:48.290 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:48.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:48.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:48.291 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:09:48.524 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4097505898,v1:127.0.0.1:6811/4097505898] [v2:127.0.0.1:6812/4097505898,v1:127.0.0.1:6813/4097505898] exists,up fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:09:48.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:48.526 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:48.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:48.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:48.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:09:48.528 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:48.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e5146ca4-4afc-4671-add8-9f7859cbc57a 2026-02-13T19:09:48.528 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e5146ca4-4afc-4671-add8-9f7859cbc57a' 2026-02-13T19:09:48.529 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 e5146ca4-4afc-4671-add8-9f7859cbc57a 2026-02-13T19:09:48.529 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:48.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD8do9pvBMQIBAAphMluSGiCepChokz8Arm4w== 2026-02-13T19:09:48.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD8do9pvBMQIBAAphMluSGiCepChokz8Arm4w=="}' 2026-02-13T19:09:48.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e5146ca4-4afc-4671-add8-9f7859cbc57a -i td/test-erasure-eio/2/new.json 2026-02-13T19:09:48.770 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:09:48.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:09:48.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD8do9pvBMQIBAAphMluSGiCepChokz8Arm4w== --osd-uuid e5146ca4-4afc-4671-add8-9f7859cbc57a 2026-02-13T19:09:48.795 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:48.793+0000 7fb7b6c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:48.796 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:48.795+0000 7fb7b6c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:48.797 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:48.796+0000 7fb7b6c48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:48.798 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:48.796+0000 7fb7b6c48780 -1 bdev(0x36eec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:48.798 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:48.796+0000 7fb7b6c48780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:09:49.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:09:49.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:49.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:09:49.107 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:09:49.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:49.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:09:49.422 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:09:49.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:49.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:49.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:49.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:49.437 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:49.435+0000 7f5564c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:49.438 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:49.436+0000 7f5564c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:49.439 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:49.437+0000 7f5564c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:49.569 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:49.567+0000 7f5564c2d780 -1 Falling back to public interface 2026-02-13T19:09:49.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:09:49.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:49.651 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:49.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:49.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:09:49.872 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:49.870+0000 7f5564c2d780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:09:49.876 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:50.877 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:50.877 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:50.878 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:50.878 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:50.879 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:50.879 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:09:51.102 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1848481247,v1:127.0.0.1:6819/1848481247] [v2:127.0.0.1:6820/1848481247,v1:127.0.0.1:6821/1848481247] exists,up e5146ca4-4afc-4671-add8-9f7859cbc57a 2026-02-13T19:09:51.106 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:09:51.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:51.108 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:51.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:09:51.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 90bf2160-b04a-4abc-a5e4-478c0fc79170' 2026-02-13T19:09:51.111 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:09:51.111 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:09:51.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD/do9phWDcBhAAyv+q1CQOZZL3pn8QuLMGTw== 2026-02-13T19:09:51.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD/do9phWDcBhAAyv+q1CQOZZL3pn8QuLMGTw=="}' 2026-02-13T19:09:51.116 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90bf2160-b04a-4abc-a5e4-478c0fc79170 -i td/test-erasure-eio/3/new.json 2026-02-13T19:09:51.344 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:09:51.352 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:09:51.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD/do9phWDcBhAAyv+q1CQOZZL3pn8QuLMGTw== --osd-uuid 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:09:51.369 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:51.367+0000 7fcb3260e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:51.370 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:51.368+0000 7fcb3260e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:51.371 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:51.369+0000 7fcb3260e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:51.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:51.370+0000 7fcb3260e780 -1 bdev(0x2836c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:09:51.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:51.370+0000 7fcb3260e780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:09:51.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:09:51.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:09:51.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:09:51.700 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:09:51.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:09:52.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:09:52.018 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:09:52.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:52.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:09:52.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:09:52.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:09:52.033 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:52.032+0000 7f695e82f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:52.035 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:52.033+0000 7f695e82f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:52.036 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:52.034+0000 7f695e82f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:09:52.164 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:52.162+0000 7f695e82f780 -1 Falling back to public interface 2026-02-13T19:09:52.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:09:52.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:09:52.245 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:09:52.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:52.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:09:52.464 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:09:52.462+0000 7f695e82f780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:09:52.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:09:53.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:09:53.472 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:09:53.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:09:53.473 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:09:53.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:09:53.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:09:53.696 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2099931123,v1:127.0.0.1:6827/2099931123] [v2:127.0.0.1:6828/2099931123,v1:127.0.0.1:6829/2099931123] exists,up 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:53.698 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:53.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:09:53.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:09:53.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:09:53.743 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:09:53.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:09:53.750 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:09:44.721+0000 7f3b659d8780 0 load: jerasure load: lrc 2026-02-13T19:09:53.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-13T19:09:53.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:09:53.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:09:54.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:09:54.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:09:54.406 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:09:54.414 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:09:55.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:09:55.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:09:55.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:09:55.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:09:55.417 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:09:55.418 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:09:55.418 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:09:55.418 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:09:55.418 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:09:55.419 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:09:55.503 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:09:55.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:09:55.727 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:55.728 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:09:55.728 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:09:55.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:09:55.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:55.728 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:09:55.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:09:55.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:09:55.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:09:55.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:55.792 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:09:55.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-13T19:09:55.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-13T19:09:55.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-13T19:09:55.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:55.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:09:55.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-13T19:09:55.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-13T19:09:55.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-13T19:09:55.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:09:55.920 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:09:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-13T19:09:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-13T19:09:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-107374182404' 2026-02-13T19:09:55.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:55.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:09:55.987 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:55.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:09:55.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:09:55.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:55.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:09:55.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:09:55.989 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:09:55.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:56.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:09:56.213 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:09:57.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:09:57.214 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:09:57.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:09:57.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:57.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-13T19:09:57.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:57.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:09:57.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-13T19:09:57.443 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:57.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-13T19:09:57.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-13T19:09:57.444 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607556 2026-02-13T19:09:57.445 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:09:57.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-13T19:09:57.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:57.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-13T19:09:57.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:57.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:09:57.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-13T19:09:57.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:57.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-13T19:09:57.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-13T19:09:57.677 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411331 2026-02-13T19:09:57.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:09:57.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-13T19:09:57.899 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:09:57.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182404 2026-02-13T19:09:57.899 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:09:57.900 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:09:57.901 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182404 2026-02-13T19:09:57.901 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:09:57.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182404 2026-02-13T19:09:57.902 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-13T19:09:57.903 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182404 2026-02-13T19:09:57.903 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:09:58.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-13T19:09:58.128 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:09:58.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:58.129 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:58.455 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:09:58.455 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:09:58.456 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:09:58.456 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:09:58.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:09:58.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:09:58.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:09:58.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:09:58.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:09:58.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:09:58.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:09:58.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:09:58.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-0-10 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-0-10 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-0-10 2026-02-13T19:09:58.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:09:58.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-0-10 td/test-erasure-eio/ORIGINAL 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-0-10 td/test-erasure-eio 0 10 set 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-10 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:09:59.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-10 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-10 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-10 2026-02-13T19:09:59.014 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-13T19:09:59.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:09:59.523 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:09:59.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:09:59.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-13T19:09:59.532 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-13T19:09:59.533 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:10 bytes copied, 3.768e-05 s, 265 kB/s 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:09:59.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:09:59.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-13T19:09:59.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:09:59.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:59.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:09:59.997 INFO:tasks.workunit.client.0.trial034.stderr:start osd.3 2026-02-13T19:09:59.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:09:59.998 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-13T19:09:59.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:09:59.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:09:59.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:09:59.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:00.009 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:00.007+0000 7f28f8833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:00.010 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:00.009+0000 7f28f8833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:00.011 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:00.010+0000 7f28f8833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:00.146 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:00.144+0000 7f28f8833780 -1 Falling back to public interface 2026-02-13T19:10:00.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:10:00.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:00.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:00.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:00.451 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:00.472 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:00.470+0000 7f28f8833780 -1 osd.3 34 log_to_monitors true 2026-02-13T19:10:01.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:01.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:01.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:01.453 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:01.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:01.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:01.678 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 up in weight 1 up_from 38 up_thru 38 down_at 35 last_clean_interval [25,34) [v2:127.0.0.1:6826/254993348,v1:127.0.0.1:6827/254993348] [v2:127.0.0.1:6828/254993348,v1:127.0.0.1:6829/254993348] exists,up 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:01.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:01.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:01.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:01.763 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:01.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:01.987 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:01.987 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:01.987 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:01.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:01.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:01.988 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:02.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-13T19:10:02.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-13T19:10:02.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-13T19:10:02.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:02.051 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-13T19:10:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-13T19:10:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-13T19:10:02.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:02.113 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:02.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-13T19:10:02.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-13T19:10:02.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411335' 2026-02-13T19:10:02.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:02.176 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:02.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-13T19:10:02.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-13T19:10:02.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411335 3-163208757251' 2026-02-13T19:10:02.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:02.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-13T19:10:02.240 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:02.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:02.241 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-13T19:10:02.241 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:02.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-13T19:10:02.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-13T19:10:02.243 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803784 2026-02-13T19:10:02.243 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:02.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-13T19:10:02.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:03.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:03.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:03.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-13T19:10:03.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:03.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-13T19:10:03.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:03.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:03.692 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-13T19:10:03.692 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:03.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-13T19:10:03.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-13T19:10:03.694 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 51539607559 2026-02-13T19:10:03.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:03.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-13T19:10:03.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:03.917 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411335 2026-02-13T19:10:03.918 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:03.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:03.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-13T19:10:03.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:03.920 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-13T19:10:03.921 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-13T19:10:03.921 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411335 2026-02-13T19:10:03.921 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:04.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-13T19:10:04.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:04.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-163208757251 2026-02-13T19:10:04.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:04.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:04.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-163208757251 2026-02-13T19:10:04.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:04.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757251 2026-02-13T19:10:04.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 163208757251' 2026-02-13T19:10:04.149 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 163208757251 2026-02-13T19:10:04.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:04.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-13T19:10:04.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:04.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:04.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:04.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:04.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:04.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:04.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:04.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:04.920 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:05.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:05.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:05.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:05.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:05.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:05.548 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-10 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-10 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:10:05.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-0-10 td/test-erasure-eio/COPY 2026-02-13T19:10:05.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:10:05.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:10:05.577 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-0-10 td/test-erasure-eio 1 10 set 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-10 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:10:05.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-10 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-10 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-10 2026-02-13T19:10:05.581 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:10:05.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:10:05.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:10:06.130 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:10:06.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:10:06.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-13T19:10:06.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:10 bytes copied, 4.6967e-05 s, 213 kB/s 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:10:06.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:06.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:06.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:06.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:10:06.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:10:06.594 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:10:06.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:06.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:10:06.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:10:06.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:06.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:06.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:06.604 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:06.603+0000 7f025380f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:06.606 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:06.605+0000 7f025380f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:06.607 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:06.606+0000 7f025380f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:06.752 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:06.751+0000 7f025380f780 -1 Falling back to public interface 2026-02-13T19:10:06.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:10:06.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:06.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:06.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:07.039 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:07.038+0000 7f025380f780 -1 osd.1 43 log_to_monitors true 2026-02-13T19:10:07.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:08.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:08.270 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [12,43) [v2:127.0.0.1:6810/384660450,v1:127.0.0.1:6811/384660450] [v2:127.0.0.1:6812/384660450,v1:127.0.0.1:6813/384660450] exists,up fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:10:08.270 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:08.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:08.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:08.583 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:08.583 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:08.583 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:08.584 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:08.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:08.584 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:08.584 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:08.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-13T19:10:08.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-13T19:10:08.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-13T19:10:08.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:08.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:08.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-13T19:10:08.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-13T19:10:08.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-210453397507' 2026-02-13T19:10:08.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:08.712 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:08.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-13T19:10:08.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-13T19:10:08.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-210453397507 2-77309411338' 2026-02-13T19:10:08.777 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:08.777 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:08.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757254 2026-02-13T19:10:08.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757254 2026-02-13T19:10:08.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-210453397507 2-77309411338 3-163208757254' 2026-02-13T19:10:08.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:08.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-13T19:10:08.840 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:08.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:08.841 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-13T19:10:08.842 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:08.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-13T19:10:08.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-13T19:10:08.843 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803787 2026-02-13T19:10:08.843 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:09.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-13T19:10:09.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:09.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397507 2026-02-13T19:10:09.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:09.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:09.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397507 2026-02-13T19:10:09.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:09.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-13T19:10:09.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-13T19:10:09.078 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 210453397507 2026-02-13T19:10:09.078 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:09.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 210453397507 2026-02-13T19:10:09.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:10.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:10.303 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:10.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 210453397507 2026-02-13T19:10:10.525 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:11.526 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:10:11.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:11.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-13T19:10:11.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:11.765 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411338 2026-02-13T19:10:11.765 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:11.765 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:11.766 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411338 2026-02-13T19:10:11.766 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:11.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411338 2026-02-13T19:10:11.767 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-13T19:10:11.767 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411338 2026-02-13T19:10:11.767 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:11.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-13T19:10:11.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:11.993 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-163208757254 2026-02-13T19:10:11.993 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:11.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:11.994 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-163208757254 2026-02-13T19:10:11.994 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:11.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757254 2026-02-13T19:10:11.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 163208757254' 2026-02-13T19:10:11.996 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 163208757254 2026-02-13T19:10:11.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:12.223 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757255 -lt 163208757254 2026-02-13T19:10:12.224 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:12.224 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:12.225 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:12.540 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:12.541 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:12.765 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:12.766 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:12.766 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:12.766 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:13.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:13.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:13.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:13.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:13.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:13.410 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:13.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-10 fail 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-10 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-13T19:10:13.421 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-94542-0-10 td/test-erasure-eio/COPY 2026-02-13T19:10:13.438 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj-size-94542-0-10: (5) Input/output error 2026-02-13T19:10:13.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-13T19:10:13.439 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:10:13.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-0-0 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-0-0 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:13.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:10:13.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-0-0 td/test-erasure-eio/ORIGINAL 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-0-0 td/test-erasure-eio 0 0 set 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-0 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-13T19:10:13.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-0 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:13.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:10:13.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:10:13.690 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:13.690 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:10:13.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:10:13.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:10:13.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:10:13.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-13T19:10:13.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:10:13.971 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:10:13.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:10:13.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-13T19:10:13.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-13T19:10:13.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:13.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:13.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:13.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-13T19:10:13.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:13.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:14.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:14.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:14.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:14.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:14.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:10:14.425 INFO:tasks.workunit.client.0.trial034.stderr:start osd.3 2026-02-13T19:10:14.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:14.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-13T19:10:14.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:10:14.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:14.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:14.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:14.437 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:14.436+0000 7f1518243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:14.438 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:14.437+0000 7f1518243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:14.439 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:14.438+0000 7f1518243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:14.569 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:14.568+0000 7f1518243780 -1 Falling back to public interface 2026-02-13T19:10:14.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:14.661 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:14.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:14.878 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:14.877+0000 7f1518243780 -1 osd.3 54 log_to_monitors true 2026-02-13T19:10:14.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:15.887 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:16.117 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [38,54) [v2:127.0.0.1:6826/1512276182,v1:127.0.0.1:6827/1512276182] [v2:127.0.0.1:6828/1512276182,v1:127.0.0.1:6829/1512276182] exists,up 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:10:16.117 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:16.118 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:16.119 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:16.119 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:16.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:16.119 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:16.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:16.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:16.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:16.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:16.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:16.427 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:16.428 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:16.428 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:16.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:16.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:16.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:16.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-13T19:10:16.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-13T19:10:16.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-13T19:10:16.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:16.493 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:16.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-13T19:10:16.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-13T19:10:16.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510' 2026-02-13T19:10:16.559 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:16.559 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:16.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411342 2026-02-13T19:10:16.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411342 2026-02-13T19:10:16.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-77309411342' 2026-02-13T19:10:16.626 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:16.626 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:16.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103171 2026-02-13T19:10:16.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103171 2026-02-13T19:10:16.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-77309411342 3-249108103171' 2026-02-13T19:10:16.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:16.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-13T19:10:16.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:16.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:16.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-13T19:10:16.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:16.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-13T19:10:16.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-13T19:10:16.697 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803791 2026-02-13T19:10:16.697 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:16.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-13T19:10:16.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:16.927 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397510 2026-02-13T19:10:16.928 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:16.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:16.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397510 2026-02-13T19:10:16.929 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:16.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-13T19:10:16.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-13T19:10:16.930 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 210453397510 2026-02-13T19:10:16.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:17.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-13T19:10:17.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:17.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411342 2026-02-13T19:10:17.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:17.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:17.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411342 2026-02-13T19:10:17.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:17.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411342 2026-02-13T19:10:17.161 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411342' 2026-02-13T19:10:17.161 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411342 2026-02-13T19:10:17.161 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:17.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411342 -lt 77309411342 2026-02-13T19:10:17.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:17.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-249108103171 2026-02-13T19:10:17.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:17.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:17.389 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-249108103171 2026-02-13T19:10:17.389 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:17.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=249108103171 2026-02-13T19:10:17.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 249108103171' 2026-02-13T19:10:17.390 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 249108103171 2026-02-13T19:10:17.390 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:17.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 249108103171 2026-02-13T19:10:17.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:18.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:18.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:18.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 249108103171 -lt 249108103171 2026-02-13T19:10:18.844 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:18.844 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:18.845 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:19.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:19.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:19.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:19.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:19.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:19.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:19.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:19.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:19.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:19.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:19.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:20.010 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-0 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:10:20.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-0-0 td/test-erasure-eio/COPY 2026-02-13T19:10:20.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:10:20.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:10:20.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-0-0 td/test-erasure-eio 1 0 set 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-0 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:10:20.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-0 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-0 2026-02-13T19:10:20.044 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:10:20.270 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:10:20.270 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:20.270 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:10:20.270 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:10:20.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:10:20.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:10:20.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:10:20.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:10:20.588 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:10:20.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:10:20.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-13T19:10:20.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-13T19:10:20.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:20.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:10:20.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:20.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:20.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:20.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:20.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:20.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:20.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:20.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:20.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:20.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:10:20.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:20.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:20.909 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:21.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:10:21.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:21.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:10:21.049 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:21.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:10:21.051 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:10:21.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:21.052 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:10:21.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:10:21.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:21.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:21.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:21.061 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:21.060+0000 7f9cab468780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:21.062 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:21.061+0000 7f9cab468780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:21.063 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:21.062+0000 7f9cab468780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:21.201 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:21.200+0000 7f9cab468780 -1 Falling back to public interface 2026-02-13T19:10:21.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:21.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:21.505 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:21.546 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:21.545+0000 7f9cab468780 -1 osd.1 63 log_to_monitors true 2026-02-13T19:10:22.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:22.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:22.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:22.507 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:22.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:22.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:22.732 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 67 up_thru 67 down_at 64 last_clean_interval [49,63) [v2:127.0.0.1:6810/1504646323,v1:127.0.0.1:6811/1504646323] [v2:127.0.0.1:6812/1504646323,v1:127.0.0.1:6813/1504646323] exists,up fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:22.733 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:22.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:22.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:22.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:22.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:22.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:22.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:22.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:22.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:22.822 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:23.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:23.055 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:23.056 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:23.056 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:23.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:23.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:23.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:23.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-13T19:10:23.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-13T19:10:23.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794' 2026-02-13T19:10:23.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:23.120 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:23.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808835 2026-02-13T19:10:23.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808835 2026-02-13T19:10:23.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-287762808835' 2026-02-13T19:10:23.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:23.183 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:23.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411345 2026-02-13T19:10:23.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411345 2026-02-13T19:10:23.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-287762808835 2-77309411345' 2026-02-13T19:10:23.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:23.247 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103174 2026-02-13T19:10:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103174 2026-02-13T19:10:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-287762808835 2-77309411345 3-249108103174' 2026-02-13T19:10:23.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:23.310 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803794 2026-02-13T19:10:23.311 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:23.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:23.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-13T19:10:23.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:23.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803794 2026-02-13T19:10:23.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-13T19:10:23.313 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803794 2026-02-13T19:10:23.314 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:23.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803794 2026-02-13T19:10:23.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:24.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:24.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:24.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-13T19:10:24.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:24.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-287762808835 2026-02-13T19:10:24.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:24.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:24.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-287762808835 2026-02-13T19:10:24.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:24.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808835 2026-02-13T19:10:24.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 287762808835' 2026-02-13T19:10:24.781 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 287762808835 2026-02-13T19:10:24.782 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:25.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-13T19:10:25.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:25.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411345 2026-02-13T19:10:25.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:25.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:25.010 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411345 2026-02-13T19:10:25.010 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:25.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411345 2026-02-13T19:10:25.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411345' 2026-02-13T19:10:25.012 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411345 2026-02-13T19:10:25.012 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:25.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411345 -lt 77309411345 2026-02-13T19:10:25.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:25.236 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-249108103174 2026-02-13T19:10:25.236 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:25.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:25.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-249108103174 2026-02-13T19:10:25.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:25.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=249108103174 2026-02-13T19:10:25.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 249108103174' 2026-02-13T19:10:25.239 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 249108103174 2026-02-13T19:10:25.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:25.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 249108103174 -lt 249108103174 2026-02-13T19:10:25.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:25.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:25.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:25.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:25.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:25.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:25.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:25.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:25.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:25.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:25.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:26.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:26.005 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:26.005 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:26.005 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:26.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:26.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:26.639 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:26.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-13T19:10:26.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-0 fail 2026-02-13T19:10:26.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:26.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:26.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-0 2026-02-13T19:10:26.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-13T19:10:26.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-13T19:10:26.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-94542-0-0 td/test-erasure-eio/COPY 2026-02-13T19:10:26.667 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj-size-94542-0-0: (5) Input/output error 2026-02-13T19:10:26.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-13T19:10:26.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:10:26.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-0-256 2026-02-13T19:10:26.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-0-256 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:10:26.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-0-256 td/test-erasure-eio/ORIGINAL 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-0-256 td/test-erasure-eio 0 256 add 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-256 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:10:26.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-256 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:26.689 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:10:26.916 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:10:26.916 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:26.916 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:10:26.916 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:10:26.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:10:26.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:10:26.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-13T19:10:26.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:10:27.198 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-13T19:10:27.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:27.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:27.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-13T19:10:27.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:27.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:27.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:27.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-13T19:10:27.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:27.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:27.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:start osd.3 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:27.610 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-13T19:10:27.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:10:27.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:27.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:27.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:27.622 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:27.621+0000 7f9ad9a69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:27.623 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:27.622+0000 7f9ad9a69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:27.624 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:27.623+0000 7f9ad9a69780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:27.776 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:27.775+0000 7f9ad9a69780 -1 Falling back to public interface 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:27.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:28.058 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:28.057+0000 7f9ad9a69780 -1 osd.3 70 log_to_monitors true 2026-02-13T19:10:28.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:29.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:29.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:29.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:29.060 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:29.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:29.061 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:29.288 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [58,70) [v2:127.0.0.1:6826/1708336414,v1:127.0.0.1:6827/1708336414] [v2:127.0.0.1:6828/1708336414,v1:127.0.0.1:6829/1708336414] exists,up 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:29.289 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:29.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:29.290 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:29.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:29.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:29.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:29.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:29.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:29.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:29.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:29.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:29.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:29.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:29.604 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:29.605 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:29.605 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:29.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:29.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:29.605 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:29.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-13T19:10:29.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-13T19:10:29.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797' 2026-02-13T19:10:29.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:29.671 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:29.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808838 2026-02-13T19:10:29.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808838 2026-02-13T19:10:29.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-287762808838' 2026-02-13T19:10:29.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:29.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:29.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411348 2026-02-13T19:10:29.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411348 2026-02-13T19:10:29.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-287762808838 2-77309411348' 2026-02-13T19:10:29.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:29.802 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:29.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514499 2026-02-13T19:10:29.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514499 2026-02-13T19:10:29.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-287762808838 2-77309411348 3-326417514499' 2026-02-13T19:10:29.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:29.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803797 2026-02-13T19:10:29.869 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:29.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:29.871 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-13T19:10:29.871 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:29.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803797 2026-02-13T19:10:29.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-13T19:10:29.872 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803797 2026-02-13T19:10:29.872 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:30.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-13T19:10:30.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:31.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:31.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:31.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-13T19:10:31.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:31.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-287762808838 2026-02-13T19:10:31.328 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:31.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:31.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-287762808838 2026-02-13T19:10:31.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:31.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808838 2026-02-13T19:10:31.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 287762808838' 2026-02-13T19:10:31.331 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 287762808838 2026-02-13T19:10:31.332 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:31.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808838 -lt 287762808838 2026-02-13T19:10:31.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:31.563 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411348 2026-02-13T19:10:31.563 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:31.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:31.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411348 2026-02-13T19:10:31.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:31.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411348 2026-02-13T19:10:31.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411348' 2026-02-13T19:10:31.566 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411348 2026-02-13T19:10:31.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:31.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411348 -lt 77309411348 2026-02-13T19:10:31.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:31.792 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-326417514499 2026-02-13T19:10:31.792 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:31.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:31.793 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-326417514499 2026-02-13T19:10:31.794 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:31.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=326417514499 2026-02-13T19:10:31.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 326417514499' 2026-02-13T19:10:31.795 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 326417514499 2026-02-13T19:10:31.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:32.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 326417514499 -lt 326417514499 2026-02-13T19:10:32.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:32.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:32.022 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:32.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:32.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:32.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:32.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:32.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:32.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:32.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:32.330 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:32.551 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:32.552 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:32.552 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:32.553 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:32.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:256 bytes copied, 1.9596e-05 s, 13.1 MB/s 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-13T19:10:32.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:32.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:33.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:33.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:33.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-13T19:10:33.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:10:33.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:33.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:start osd.3 2026-02-13T19:10:33.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:33.314 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-13T19:10:33.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-13T19:10:33.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:33.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:33.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:33.324 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:33.323+0000 7fbd7d843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:33.326 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:33.325+0000 7fbd7d843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:33.327 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:33.326+0000 7fbd7d843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:33.470 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:33.470+0000 7fbd7d843780 -1 Falling back to public interface 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:33.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:33.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:33.536 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:33.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:33.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:33.757 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:33.764 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:33.763+0000 7fbd7d843780 -1 osd.3 77 log_to_monitors true 2026-02-13T19:10:34.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:34.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:34.759 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:34.760 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:34.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:34.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:osd.3 up in weight 1 up_from 81 up_thru 81 down_at 78 last_clean_interval [76,77) [v2:127.0.0.1:6826/2218617854,v1:127.0.0.1:6827/2218617854] [v2:127.0.0.1:6828/2218617854,v1:127.0.0.1:6829/2218617854] exists,up 90bf2160-b04a-4abc-a5e4-478c0fc79170 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:34.994 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:34.995 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:34.995 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:34.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:34.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:34.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:35.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:35.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:35.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:35.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:35.080 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:35.080 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:35.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:35.312 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:35.312 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:35.313 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:35.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:35.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:35.313 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:35.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803800 2026-02-13T19:10:35.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803800 2026-02-13T19:10:35.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803800' 2026-02-13T19:10:35.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:35.378 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:35.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808841 2026-02-13T19:10:35.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808841 2026-02-13T19:10:35.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803800 1-287762808841' 2026-02-13T19:10:35.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:35.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:35.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411351 2026-02-13T19:10:35.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411351 2026-02-13T19:10:35.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803800 1-287762808841 2-77309411351' 2026-02-13T19:10:35.507 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:35.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:35.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350979 2026-02-13T19:10:35.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350979 2026-02-13T19:10:35.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803800 1-287762808841 2-77309411351 3-347892350979' 2026-02-13T19:10:35.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:35.572 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803800 2026-02-13T19:10:35.572 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:35.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:35.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803800 2026-02-13T19:10:35.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:35.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803800 2026-02-13T19:10:35.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803800' 2026-02-13T19:10:35.574 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803800 2026-02-13T19:10:35.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:35.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803800 2026-02-13T19:10:35.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:36.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:36.800 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:37.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803800 2026-02-13T19:10:37.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:37.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-287762808841 2026-02-13T19:10:37.033 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:37.033 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:37.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-287762808841 2026-02-13T19:10:37.034 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:37.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808841 2026-02-13T19:10:37.035 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 287762808841' 2026-02-13T19:10:37.035 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 287762808841 2026-02-13T19:10:37.036 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:37.263 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808842 -lt 287762808841 2026-02-13T19:10:37.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:37.264 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411351 2026-02-13T19:10:37.264 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:37.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:37.266 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411351 2026-02-13T19:10:37.266 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:37.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411351 2026-02-13T19:10:37.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411351' 2026-02-13T19:10:37.267 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411351 2026-02-13T19:10:37.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:37.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411352 -lt 77309411351 2026-02-13T19:10:37.491 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:37.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-347892350979 2026-02-13T19:10:37.492 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:37.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:37.494 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-347892350979 2026-02-13T19:10:37.494 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:37.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350979 2026-02-13T19:10:37.495 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 347892350979' 2026-02-13T19:10:37.495 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 347892350979 2026-02-13T19:10:37.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:37.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350979 -lt 347892350979 2026-02-13T19:10:37.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:37.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:37.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:38.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:38.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:38.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:38.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:38.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:38.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:38.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:38.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:38.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:38.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:38.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:38.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:38.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:38.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:38.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:38.586 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:38.587 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:38.924 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-256 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:10:38.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-0-256 td/test-erasure-eio/COPY 2026-02-13T19:10:38.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:10:38.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:10:38.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-13T19:10:38.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-13T19:10:38.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-0-256 td/test-erasure-eio 1 256 add 2026-02-13T19:10:38.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-0-256 2026-02-13T19:10:38.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-0-256 2026-02-13T19:10:38.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-0-256 2026-02-13T19:10:38.956 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:10:39.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:10:39.180 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:39.180 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:10:39.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:10:39.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:10:39.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:10:39.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:10:39.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:10:39.491 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:10:39.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:39.502 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:39.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:39.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:39.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:10:39.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:39.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:39.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:39.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:39.892 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:10:39.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:10:39.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:39.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:39.893 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:39.902 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:39.902+0000 7fd9c444a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:39.904 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:39.903+0000 7fd9c444a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:39.905 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:39.904+0000 7fd9c444a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:40.040 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:40.039+0000 7fd9c444a780 -1 Falling back to public interface 2026-02-13T19:10:40.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:10:40.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:40.118 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:10:40.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:40.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:40.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:40.119 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:40.119 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:40.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:40.120 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:40.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:40.356 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:40.355+0000 7fd9c444a780 -1 osd.1 85 log_to_monitors true 2026-02-13T19:10:41.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:41.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:41.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:41.345 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:41.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:41.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 91 up_thru 91 down_at 88 last_clean_interval [67,85) [v2:127.0.0.1:6810/1557784592,v1:127.0.0.1:6811/1557784592] [v2:127.0.0.1:6812/1557784592,v1:127.0.0.1:6813/1557784592] exists,up fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:41.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:41.660 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:41.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:41.886 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:41.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-13T19:10:41.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-13T19:10:41.948 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804' 2026-02-13T19:10:41.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:41.949 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:42.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=390842023939 2026-02-13T19:10:42.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 390842023939 2026-02-13T19:10:42.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-390842023939' 2026-02-13T19:10:42.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:42.014 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:42.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411355 2026-02-13T19:10:42.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411355 2026-02-13T19:10:42.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-390842023939 2-77309411355' 2026-02-13T19:10:42.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:42.079 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:42.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350982 2026-02-13T19:10:42.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350982 2026-02-13T19:10:42.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-390842023939 2-77309411355 3-347892350982' 2026-02-13T19:10:42.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:42.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803804 2026-02-13T19:10:42.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:42.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:42.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-13T19:10:42.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:42.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803804 2026-02-13T19:10:42.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-13T19:10:42.147 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803804 2026-02-13T19:10:42.147 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:42.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803804 2026-02-13T19:10:42.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:43.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:43.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:43.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-13T19:10:43.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:43.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-390842023939 2026-02-13T19:10:43.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:43.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:43.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-390842023939 2026-02-13T19:10:43.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:43.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=390842023939 2026-02-13T19:10:43.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 390842023939' 2026-02-13T19:10:43.604 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 390842023939 2026-02-13T19:10:43.604 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:43.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-13T19:10:43.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:43.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411355 2026-02-13T19:10:43.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:43.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:43.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411355 2026-02-13T19:10:43.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:43.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411355 2026-02-13T19:10:43.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411355' 2026-02-13T19:10:43.829 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411355 2026-02-13T19:10:43.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:44.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411355 -lt 77309411355 2026-02-13T19:10:44.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:44.052 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-347892350982 2026-02-13T19:10:44.053 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:44.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:44.054 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-347892350982 2026-02-13T19:10:44.054 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:44.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350982 2026-02-13T19:10:44.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 347892350982' 2026-02-13T19:10:44.055 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 347892350982 2026-02-13T19:10:44.055 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:44.281 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350982 -lt 347892350982 2026-02-13T19:10:44.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:44.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:44.282 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:44.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:44.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:44.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:44.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:44.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:44.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:45.137 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:45.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:45.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:45.138 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:256 bytes copied, 2.2172e-05 s, 11.5 MB/s 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:10:45.140 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:45.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:45.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:45.246 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:10:45.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:10:45.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:45.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:45.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:10:45.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:10:45.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:10:45.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:10:45.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:10:45.398 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:45.396+0000 7f01f580f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:45.399 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:45.398+0000 7f01f580f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:45.400 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:45.399+0000 7f01f580f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:45.524 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:45.523+0000 7f01f580f780 -1 Falling back to public interface 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:45.611 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:10:45.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:45.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:45.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:45.799+0000 7f01f580f780 -1 osd.1 92 log_to_monitors true 2026-02-13T19:10:45.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:46.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:46.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:46.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:46.844 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:46.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:46.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 96 up_thru 96 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6810/3621524601,v1:127.0.0.1:6811/3621524601] [v2:127.0.0.1:6812/3621524601,v1:127.0.0.1:6813/3621524601] exists,up fc346775-6cc9-48ef-82e2-6c58bbc7a88a 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:10:47.070 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:10:47.071 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:10:47.071 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:10:47.072 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:10:47.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:10:47.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:10:47.072 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:10:47.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:10:47.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:10:47.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:10:47.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:10:47.160 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:10:47.160 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:47.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:10:47.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-13T19:10:47.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-13T19:10:47.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807' 2026-02-13T19:10:47.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:47.448 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:10:47.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860419 2026-02-13T19:10:47.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860419 2026-02-13T19:10:47.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-412316860419' 2026-02-13T19:10:47.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:47.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:10:47.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411358 2026-02-13T19:10:47.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411358 2026-02-13T19:10:47.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-412316860419 2-77309411358' 2026-02-13T19:10:47.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:10:47.577 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:10:47.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350985 2026-02-13T19:10:47.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350985 2026-02-13T19:10:47.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-412316860419 2-77309411358 3-347892350985' 2026-02-13T19:10:47.640 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:47.641 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803807 2026-02-13T19:10:47.641 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:47.641 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:10:47.642 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-13T19:10:47.642 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:47.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803807 2026-02-13T19:10:47.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-13T19:10:47.643 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803807 2026-02-13T19:10:47.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:47.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-13T19:10:47.868 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:10:48.869 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:10:48.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:10:49.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-13T19:10:49.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:49.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-412316860419 2026-02-13T19:10:49.101 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:49.101 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:10:49.101 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-412316860419 2026-02-13T19:10:49.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:49.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=412316860419 2026-02-13T19:10:49.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 412316860419' 2026-02-13T19:10:49.103 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 412316860419 2026-02-13T19:10:49.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:10:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-13T19:10:49.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:49.331 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411358 2026-02-13T19:10:49.331 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:49.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:10:49.332 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411358 2026-02-13T19:10:49.332 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:49.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411358 2026-02-13T19:10:49.333 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411358' 2026-02-13T19:10:49.334 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411358 2026-02-13T19:10:49.334 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:10:49.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411358 -lt 77309411358 2026-02-13T19:10:49.560 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:10:49.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-347892350985 2026-02-13T19:10:49.561 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:10:49.561 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:10:49.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-347892350985 2026-02-13T19:10:49.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:10:49.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350985 2026-02-13T19:10:49.563 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 347892350985' 2026-02-13T19:10:49.563 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 347892350985 2026-02-13T19:10:49.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:10:49.796 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350985 -lt 347892350985 2026-02-13T19:10:49.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:10:49.797 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:49.798 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:10:50.108 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:10:50.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:10:50.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:10:50.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:10:50.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:10:50.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:10:50.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:10:50.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:10:50.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:10:50.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:10:50.646 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:10:50.985 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-0-256 fail 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-0-256 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-13T19:10:50.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-94542-0-256 td/test-erasure-eio/COPY 2026-02-13T19:10:51.010 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj-size-94542-0-256: (5) Input/output error 2026-02-13T19:10:51.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-13T19:10:51.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:10:51.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:10:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:10:51.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:10:51.249 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:10:51.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:10:51.535 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:10:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:51.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:51.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:10:51.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:10:51.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:10:51.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:10:51.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:10:51.657 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:10:51.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:10:51.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:10:51.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:10:51.659 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:10:51.659 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:10:51.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:10:51.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:10:51.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:10:51.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:10:51.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:10:51.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:10:51.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:10:51.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:10:51.681 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:10:51.681 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:10:51.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:10:51.682 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:10:51.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:10:51.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:10:51.683 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:10:51.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:10:51.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:10:51.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:10:51.685 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:10:51.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:10:51.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:10:51.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:10:51.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:10:51.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:10:51.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:10:51.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:10:51.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:10:51.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:10:51.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:10:51.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:10:51.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:10:51.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:10:51.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:51.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:10:51.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:10:51.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:10:51.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:10:51.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:10:51.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:10:51.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:10:51.736 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:10:51.736 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:10:51.737 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:10:51.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:10:51.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:10:51.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:10:51.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:10:51.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:10:51.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:10:51.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:10:51.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:10:51.782 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:10:51.782 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.783 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.783 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:10:51.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:10:51.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:10:51.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:10:51.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:10:51.922 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:51.923 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:51.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:10:51.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:10:51.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:10:51.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:10:52.033 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:10:52.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:10:53.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:10:53.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:10:53.081 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:10:53.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:10:53.087 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:10:51.726+0000 7f78372fbc00 0 load: jerasure load: lrc 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:10:53.088 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:10:53.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:10:53.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:53.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:53.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:10:53.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:10:53.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3b43578d-a569-4084-a9cf-b113f98b4bc8 2026-02-13T19:10:53.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3b43578d-a569-4084-a9cf-b113f98b4bc8' 2026-02-13T19:10:53.097 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 3b43578d-a569-4084-a9cf-b113f98b4bc8 2026-02-13T19:10:53.097 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:10:53.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA9d49pNQ0oBhAANX36CE++el4elnjtAXRy1w== 2026-02-13T19:10:53.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA9d49pNQ0oBhAANX36CE++el4elnjtAXRy1w=="}' 2026-02-13T19:10:53.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3b43578d-a569-4084-a9cf-b113f98b4bc8 -i td/test-erasure-eio/0/new.json 2026-02-13T19:10:53.207 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:10:53.214 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:10:53.215 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA9d49pNQ0oBhAANX36CE++el4elnjtAXRy1w== --osd-uuid 3b43578d-a569-4084-a9cf-b113f98b4bc8 2026-02-13T19:10:53.231 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.229+0000 7f8c69835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.232 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.230+0000 7f8c69835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.233 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.231+0000 7f8c69835780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.234 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.232+0000 7f8c69835780 -1 bdev(0x2ed4c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:10:53.234 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.232+0000 7f8c69835780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:10:53.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:10:53.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:10:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:10:53.612 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:10:53.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:10:53.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:10:53.714 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:10:53.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:53.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:10:53.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:10:53.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:10:53.729 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.727+0000 7f1190c1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.731 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.729+0000 7f1190c1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.731 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.730+0000 7f1190c1b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:53.812 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:10:53.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:53.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:10:53.913 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:53.921 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:53.920+0000 7f1190c1b780 -1 Falling back to public interface 2026-02-13T19:10:54.181 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:54.179+0000 7f1190c1b780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:10:54.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:54.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:54.915 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:10:54.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:54.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:54.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:10:55.151 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/253885993,v1:127.0.0.1:6803/253885993] [v2:127.0.0.1:6804/253885993,v1:127.0.0.1:6805/253885993] exists,up 3b43578d-a569-4084-a9cf-b113f98b4bc8 2026-02-13T19:10:55.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:55.155 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:55.156 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:55.157 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:55.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 be1c470f-214b-41eb-a43d-2aa702a035ff' 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:10:55.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:10:55.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/d49paITICRAAYmQzPUBdhS9PAmWRtCxQNQ== 2026-02-13T19:10:55.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/d49paITICRAAYmQzPUBdhS9PAmWRtCxQNQ=="}' 2026-02-13T19:10:55.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new be1c470f-214b-41eb-a43d-2aa702a035ff -i td/test-erasure-eio/1/new.json 2026-02-13T19:10:55.401 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:10:55.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:10:55.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA/d49paITICRAAYmQzPUBdhS9PAmWRtCxQNQ== --osd-uuid be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:10:55.425 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:55.423+0000 7f5bbe7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:55.426 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:55.425+0000 7f5bbe7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:55.427 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:55.426+0000 7f5bbe7d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:55.428 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:55.426+0000 7f5bbe7d8780 -1 bdev(0x2d20c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:10:55.428 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:55.426+0000 7f5bbe7d8780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:10:55.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:10:55.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:10:55.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:10:55.746 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:10:55.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:10:56.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:10:56.058 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:10:56.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:56.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:10:56.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:10:56.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:10:56.072 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:56.070+0000 7f91d2469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:56.074 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:56.072+0000 7f91d2469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:56.074 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:56.073+0000 7f91d2469780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:56.212 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:56.211+0000 7f91d2469780 -1 Falling back to public interface 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:56.285 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:56.286 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:10:56.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:56.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:56.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:56.522 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:56.520+0000 7f91d2469780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:10:57.513 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:10:57.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:57.514 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:10:57.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:10:57.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:57.515 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:10:57.749 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3072434818,v1:127.0.0.1:6811/3072434818] [v2:127.0.0.1:6812/3072434818,v1:127.0.0.1:6813/3072434818] exists,up be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:10:57.753 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:10:57.754 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:10:57.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:10:57.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 04183257-f772-4845-991f-3c44432cac4d' 2026-02-13T19:10:57.757 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:10:57.757 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:10:57.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBBd49pPXloLRAA8pRzP8gAek3jPx8zQpAVkw== 2026-02-13T19:10:57.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBBd49pPXloLRAA8pRzP8gAek3jPx8zQpAVkw=="}' 2026-02-13T19:10:57.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 04183257-f772-4845-991f-3c44432cac4d -i td/test-erasure-eio/2/new.json 2026-02-13T19:10:57.996 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:10:58.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:10:58.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBd49pPXloLRAA8pRzP8gAek3jPx8zQpAVkw== --osd-uuid 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:10:58.020 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.019+0000 7fd58aa35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.022 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.020+0000 7fd58aa35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.021+0000 7fd58aa35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.021+0000 7fd58aa35780 -1 bdev(0x39ecc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:10:58.023 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.021+0000 7fd58aa35780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:10:58.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:10:58.335 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:10:58.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:10:58.336 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:10:58.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:10:58.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:10:58.649 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:10:58.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:10:58.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:10:58.650 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:10:58.651 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:10:58.664 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.662+0000 7fe8c33d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.666 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.664+0000 7fe8c33d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.666 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.665+0000 7fe8c33d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:10:58.811 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:58.809+0000 7fe8c33d8780 -1 Falling back to public interface 2026-02-13T19:10:58.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:10:58.884 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:10:58.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:10:58.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:10:58.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:10:59.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:10:59.109 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:10:59.107+0000 7fe8c33d8780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:11:00.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:00.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:00.110 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:11:00.110 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:00.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:00.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:00.333 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3134577847,v1:127.0.0.1:6819/3134577847] [v2:127.0.0.1:6820/3134577847,v1:127.0.0.1:6821/3134577847] exists,up 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:11:00.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:00.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:00.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:00.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:11:00.340 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:11:00.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=53aa6ad2-5baa-444f-ae40-05c60cbf5720 2026-02-13T19:11:00.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 53aa6ad2-5baa-444f-ae40-05c60cbf5720' 2026-02-13T19:11:00.341 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 53aa6ad2-5baa-444f-ae40-05c60cbf5720 2026-02-13T19:11:00.341 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:11:00.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEd49p53OpFBAAEJcVsChXD+FMQ88V6TfUTQ== 2026-02-13T19:11:00.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEd49p53OpFBAAEJcVsChXD+FMQ88V6TfUTQ=="}' 2026-02-13T19:11:00.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 53aa6ad2-5baa-444f-ae40-05c60cbf5720 -i td/test-erasure-eio/3/new.json 2026-02-13T19:11:00.583 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:11:00.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:11:00.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEd49p53OpFBAAEJcVsChXD+FMQ88V6TfUTQ== --osd-uuid 53aa6ad2-5baa-444f-ae40-05c60cbf5720 2026-02-13T19:11:00.608 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:00.606+0000 7fd11722d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:00.609 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:00.608+0000 7fd11722d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:00.610 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:00.608+0000 7fd11722d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:00.611 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:00.609+0000 7fd11722d780 -1 bdev(0x2bbec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:11:00.611 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:00.609+0000 7fd11722d780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:11:00.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:11:00.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:11:00.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:11:00.933 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:11:00.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:11:01.253 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:11:01.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:11:01.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:01.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:11:01.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:11:01.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:11:01.268 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:01.266+0000 7f4e13211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:01.270 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:01.268+0000 7f4e13211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:01.271 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:01.269+0000 7f4e13211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:01.404 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:01.404+0000 7f4e13211780 -1 Falling back to public interface 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:01.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:01.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:01.480 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:11:01.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:01.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:11:01.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:01.760 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:01.759+0000 7f4e13211780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:11:02.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:02.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:02.706 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:11:02.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:02.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:02.706 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:11:02.931 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1785111432,v1:127.0.0.1:6827/1785111432] [v2:127.0.0.1:6828/1785111432,v1:127.0.0.1:6829/1785111432] exists,up 53aa6ad2-5baa-444f-ae40-05c60cbf5720 2026-02-13T19:11:02.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:11:02.933 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:11:02.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:11:02.978 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:11:02.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:11:02.984 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:10:53.922+0000 7f1190c1b780 0 load: jerasure load: lrc 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:11:02.985 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:11:02.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:11:02.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:11:02.986 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:11:03.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:11:03.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:11:03.625 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:11:03.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:11:04.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:11:04.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:04.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:04.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:04.636 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:04.637 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:04.637 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:04.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:04.637 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:04.638 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:04.725 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:04.953 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:05.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:11:05.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:11:05.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:11:05.021 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:05.021 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:05.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-13T19:11:05.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-13T19:11:05.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-13T19:11:05.089 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:05.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:05.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-13T19:11:05.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-13T19:11:05.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332' 2026-02-13T19:11:05.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:05.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:05.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-13T19:11:05.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-13T19:11:05.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332 3-111669149699' 2026-02-13T19:11:05.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:05.217 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:11:05.218 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:05.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:05.218 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:11:05.219 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:05.219 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:11:05.220 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:11:05.220 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:11:05.220 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:05.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:11:05.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:06.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:06.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:06.684 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:11:06.685 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:06.685 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-13T19:11:06.685 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:06.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:06.686 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-13T19:11:06.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:06.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-13T19:11:06.688 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-13T19:11:06.688 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574852 2026-02-13T19:11:06.688 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:06.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-13T19:11:06.912 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:06.913 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-13T19:11:06.913 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:06.914 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:06.914 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-13T19:11:06.914 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:06.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-13T19:11:06.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-13T19:11:06.916 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411332 2026-02-13T19:11:06.916 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:07.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-13T19:11:07.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:07.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-13T19:11:07.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:07.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:07.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-13T19:11:07.146 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:07.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-13T19:11:07.147 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-13T19:11:07.148 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149699 2026-02-13T19:11:07.148 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:07.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-13T19:11:07.370 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:07.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:07.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:07.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:07.686 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:07.686 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:07.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:07.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:07.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:07.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:07.687 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:07.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:07.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:07.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:07.912 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:11:08.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-1-10 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:11:08.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-1-10 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:11:08.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-1-10 td/test-erasure-eio/ORIGINAL 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-1-10 td/test-erasure-eio 1 10 set 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-10 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-13T19:11:08.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-10 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:08.252 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:08.485 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:11:08.486 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:08.767 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:08.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:11:08.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-13T19:11:08.776 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-13T19:11:08.778 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:11:08.778 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:10 bytes copied, 3.735e-05 s, 268 kB/s 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:11:08.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:08.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:09.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:09.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:09.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:09.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:09.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:11:09.240 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:11:09.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:09.241 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:11:09.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:11:09.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:09.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:09.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:09.252 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:09.251+0000 7ff6739c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:09.254 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:09.253+0000 7ff6739c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:09.254 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:09.254+0000 7ff6739c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:09.380 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:09.379+0000 7ff6739c6780 -1 Falling back to public interface 2026-02-13T19:11:09.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:09.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:09.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:09.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:09.746 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:09.745+0000 7ff6739c6780 -1 osd.1 36 log_to_monitors true 2026-02-13T19:11:10.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:10.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:10.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:10.696 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:10.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:10.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:10.924 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [13,36) [v2:127.0.0.1:6810/1233162911,v1:127.0.0.1:6811/1233162911] [v2:127.0.0.1:6812/1233162911,v1:127.0.0.1:6813/1233162911] exists,up be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:10.925 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:10.926 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:10.926 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:10.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:10.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:10.927 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:11.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:11.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:11.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:11.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:11.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:11.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:11.237 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:11.237 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:11.237 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:11.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:11.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:11.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:11.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-13T19:11:11.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-13T19:11:11.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-13T19:11:11.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:11.302 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:11.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659139 2026-02-13T19:11:11.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659139 2026-02-13T19:11:11.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139' 2026-02-13T19:11:11.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:11.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:11.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-13T19:11:11.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-13T19:11:11.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139 2-77309411336' 2026-02-13T19:11:11.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:11.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:11.497 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-13T19:11:11.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-13T19:11:11.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139 2-77309411336 3-111669149702' 2026-02-13T19:11:11.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:11.498 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-13T19:11:11.498 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:11.499 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:11.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-13T19:11:11.500 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:11.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-13T19:11:11.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-13T19:11:11.501 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803784 2026-02-13T19:11:11.501 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:11.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-13T19:11:11.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:12.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:12.729 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-13T19:11:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:12.957 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659139 2026-02-13T19:11:12.957 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:12.958 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:12.959 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659139 2026-02-13T19:11:12.959 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:12.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659139 2026-02-13T19:11:12.960 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659139' 2026-02-13T19:11:12.960 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 176093659139 2026-02-13T19:11:12.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:13.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-13T19:11:13.187 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:13.188 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-13T19:11:13.188 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:13.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:13.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-13T19:11:13.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:13.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-13T19:11:13.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-13T19:11:13.191 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411336 2026-02-13T19:11:13.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:13.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-13T19:11:13.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:13.419 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-13T19:11:13.419 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:13.420 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:13.420 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-13T19:11:13.421 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:13.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-13T19:11:13.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-13T19:11:13.422 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149702 2026-02-13T19:11:13.422 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:13.660 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-13T19:11:13.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:13.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:13.661 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:13.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:13.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:13.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:13.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:13.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:13.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:13.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:13.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:14.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:14.196 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:14.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:14.197 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:14.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:14.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:14.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:14.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:11:14.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:11:14.828 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-10 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:11:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:11:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-10 td/test-erasure-eio/COPY 2026-02-13T19:11:14.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:11:14.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:11:14.859 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-1-10 td/test-erasure-eio 2 10 set 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-10 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:14.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-10 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:14.863 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-13T19:11:15.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:15.415 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:15.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:11:15.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-13T19:11:15.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-13T19:11:15.428 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:11:15.428 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:11:15.428 INFO:tasks.workunit.client.0.trial034.stderr:10 bytes copied, 4.5044e-05 s, 222 kB/s 2026-02-13T19:11:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:15.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:15.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:15.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:15.430 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:15.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:15.735 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:15.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-94542-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:15.887 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:15.888 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:15.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr:start osd.2 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:15.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:11:15.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:11:15.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:15.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:15.891 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:15.902 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:15.901+0000 7f6b6f1d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:15.903 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:15.902+0000 7f6b6f1d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:15.904 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:15.903+0000 7f6b6f1d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:16.036 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:16.035+0000 7f6b6f1d4780 -1 Falling back to public interface 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:16.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:16.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:16.330 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:16.329+0000 7f6b6f1d4780 -1 osd.2 45 log_to_monitors true 2026-02-13T19:11:16.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:17.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:17.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:17.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:17.365 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:17.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:17.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:17.601 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 up in weight 1 up_from 51 up_thru 38 down_at 48 last_clean_interval [18,45) [v2:127.0.0.1:6818/2672810592,v1:127.0.0.1:6819/2672810592] [v2:127.0.0.1:6820/2672810592,v1:127.0.0.1:6821/2672810592] exists,up 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:11:17.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:17.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:17.603 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:17.603 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:17.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:17.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:17.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:17.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:17.916 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-13T19:11:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-13T19:11:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-13T19:11:17.979 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:17.979 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:18.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659142 2026-02-13T19:11:18.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659142 2026-02-13T19:11:18.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-176093659142' 2026-02-13T19:11:18.046 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:18.046 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:18.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-13T19:11:18.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-13T19:11:18.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-176093659142 2-219043332099' 2026-02-13T19:11:18.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:18.112 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:18.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-13T19:11:18.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-13T19:11:18.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-176093659142 2-219043332099 3-111669149706' 2026-02-13T19:11:18.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:18.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-13T19:11:18.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:18.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:18.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-13T19:11:18.177 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:18.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-13T19:11:18.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-13T19:11:18.178 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803787 2026-02-13T19:11:18.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:18.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-13T19:11:18.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:19.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:19.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:19.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-13T19:11:19.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:20.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:11:20.631 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:20.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-13T19:11:20.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:20.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659142 2026-02-13T19:11:20.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:20.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:20.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659142 2026-02-13T19:11:20.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:20.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659142 2026-02-13T19:11:20.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659142' 2026-02-13T19:11:20.865 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 176093659142 2026-02-13T19:11:20.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659143 -lt 176093659142 2026-02-13T19:11:21.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:21.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-219043332099 2026-02-13T19:11:21.093 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:21.094 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:21.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-219043332099 2026-02-13T19:11:21.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:21.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-13T19:11:21.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 219043332099' 2026-02-13T19:11:21.096 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 219043332099 2026-02-13T19:11:21.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:21.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332099 -lt 219043332099 2026-02-13T19:11:21.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:21.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149706 2026-02-13T19:11:21.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:21.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:21.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149706 2026-02-13T19:11:21.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:21.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149706 2026-02-13T19:11:21.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149706' 2026-02-13T19:11:21.327 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149706 2026-02-13T19:11:21.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:21.554 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149706 -lt 111669149706 2026-02-13T19:11:21.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:21.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:21.555 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:21.889 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:21.890 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:22.115 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:22.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:22.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:22.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:22.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:22.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:22.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:22.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:11:22.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:11:22.754 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:11:22.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-10 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-10 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:11:22.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-10 td/test-erasure-eio/COPY 2026-02-13T19:11:22.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:11:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:11:22.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:11:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-13T19:11:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:11:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-1-0 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-1-0 2026-02-13T19:11:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:11:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-1-0 td/test-erasure-eio/ORIGINAL 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-1-0 td/test-erasure-eio 1 0 set 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-0 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-13T19:11:22.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-0 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:22.803 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:11:23.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:23.352 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:23.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:11:23.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-13T19:11:23.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-13T19:11:23.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:23.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:23.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:23.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:23.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:23.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:23.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:23.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:23.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:11:23.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:23.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:23.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:23.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:23.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:23.815 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:11:23.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:11:23.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:23.816 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:23.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:23.827 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:23.826+0000 7fd6d2e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:23.828 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:23.827+0000 7fd6d2e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:23.829 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:23.828+0000 7fd6d2e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:23.955 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:23.954+0000 7fd6d2e48780 -1 Falling back to public interface 2026-02-13T19:11:24.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:24.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:24.241 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:24.240+0000 7fd6d2e48780 -1 osd.1 56 log_to_monitors true 2026-02-13T19:11:24.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:25.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:25.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:25.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:25.271 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:25.271 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:25.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:25.503 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 61 up_thru 61 down_at 58 last_clean_interval [41,56) [v2:127.0.0.1:6810/2154945131,v1:127.0.0.1:6811/2154945131] [v2:127.0.0.1:6812/2154945131,v1:127.0.0.1:6813/2154945131] exists,up be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:11:25.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:25.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:25.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:25.503 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:25.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:25.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:25.504 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:25.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:25.505 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:25.505 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:25.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:25.593 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:25.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:25.817 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:25.818 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:25.818 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:25.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:25.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:25.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:25.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-13T19:11:25.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-13T19:11:25.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-13T19:11:25.883 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:25.883 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:25.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005059 2026-02-13T19:11:25.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005059 2026-02-13T19:11:25.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-261993005059' 2026-02-13T19:11:25.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:25.949 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:26.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332102 2026-02-13T19:11:26.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332102 2026-02-13T19:11:26.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-261993005059 2-219043332102' 2026-02-13T19:11:26.013 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:26.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:26.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149709 2026-02-13T19:11:26.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149709 2026-02-13T19:11:26.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-261993005059 2-219043332102 3-111669149709' 2026-02-13T19:11:26.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:26.080 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-13T19:11:26.080 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:26.080 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:26.081 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-13T19:11:26.081 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:26.082 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-13T19:11:26.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-13T19:11:26.083 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803791 2026-02-13T19:11:26.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:26.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-13T19:11:26.310 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:26.311 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-261993005059 2026-02-13T19:11:26.311 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:26.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:26.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-261993005059 2026-02-13T19:11:26.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:26.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=261993005059 2026-02-13T19:11:26.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 261993005059' 2026-02-13T19:11:26.314 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 261993005059 2026-02-13T19:11:26.314 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:26.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 261993005059 2026-02-13T19:11:26.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:27.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:27.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:27.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 261993005059 2026-02-13T19:11:27.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:28.771 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:11:28.772 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:28.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 261993005059 -lt 261993005059 2026-02-13T19:11:28.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:28.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-219043332102 2026-02-13T19:11:28.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:28.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:28.997 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-219043332102 2026-02-13T19:11:28.997 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:28.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332102 2026-02-13T19:11:28.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 219043332102' 2026-02-13T19:11:28.998 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 219043332102 2026-02-13T19:11:28.999 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:29.222 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332103 -lt 219043332102 2026-02-13T19:11:29.222 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:29.223 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149709 2026-02-13T19:11:29.223 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:29.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:29.224 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149709 2026-02-13T19:11:29.225 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:29.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149709 2026-02-13T19:11:29.226 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149709' 2026-02-13T19:11:29.226 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149709 2026-02-13T19:11:29.226 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:29.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149709 -lt 111669149709 2026-02-13T19:11:29.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:29.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:29.457 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:29.774 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:29.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:29.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:29.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:29.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:30.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:30.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:30.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:30.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:11:30.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:11:30.633 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:11:30.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-0 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:11:30.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-0 td/test-erasure-eio/COPY 2026-02-13T19:11:30.662 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:11:30.663 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:11:30.664 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-13T19:11:30.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-13T19:11:30.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-1-0 td/test-erasure-eio 2 0 set 2026-02-13T19:11:30.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-0 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:30.667 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-0 2026-02-13T19:11:30.668 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:30.668 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:30.893 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-13T19:11:30.894 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:31.237 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:31.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-13T19:11:31.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-13T19:11:31.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-13T19:11:31.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:31.250 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:31.251 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:31.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:31.252 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:31.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:31.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:31.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:31.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:31.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-13T19:11:31.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:31.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:31.558 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-94542-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:31.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:31.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:31.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr:start osd.2 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:31.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:11:31.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:11:31.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:31.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:31.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:31.704 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:31.703+0000 7f071866d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:31.705 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:31.704+0000 7f071866d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:31.706 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:31.705+0000 7f071866d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:31.823 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:31.822+0000 7f071866d780 -1 Falling back to public interface 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:31.915 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:31.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:31.916 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:31.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:31.916 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:32.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:32.191 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:32.190+0000 7f071866d780 -1 osd.2 63 log_to_monitors true 2026-02-13T19:11:33.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:33.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:33.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:33.145 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:33.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:33.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:33.374 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 up in weight 1 up_from 71 up_thru 58 down_at 68 last_clean_interval [51,63) [v2:127.0.0.1:6818/2839661057,v1:127.0.0.1:6819/2839661057] [v2:127.0.0.1:6820/2839661057,v1:127.0.0.1:6821/2839661057] exists,up 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:11:33.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:33.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:33.376 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:33.376 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:33.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:33.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:33.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:33.463 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:33.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:33.690 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:33.690 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:33.690 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:33.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:33.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:33.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:33.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-13T19:11:33.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-13T19:11:33.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794' 2026-02-13T19:11:33.756 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:33.756 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:33.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=261993005062 2026-02-13T19:11:33.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 261993005062 2026-02-13T19:11:33.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-261993005062' 2026-02-13T19:11:33.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:33.819 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:33.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678019 2026-02-13T19:11:33.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678019 2026-02-13T19:11:33.884 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-261993005062 2-304942678019' 2026-02-13T19:11:33.885 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:33.885 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:33.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149713 2026-02-13T19:11:33.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149713 2026-02-13T19:11:33.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-261993005062 2-304942678019 3-111669149713' 2026-02-13T19:11:33.949 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:33.950 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803794 2026-02-13T19:11:33.950 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:33.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:33.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-13T19:11:33.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:33.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803794 2026-02-13T19:11:33.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-13T19:11:33.952 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803794 2026-02-13T19:11:33.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:34.177 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-13T19:11:34.178 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:34.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-261993005062 2026-02-13T19:11:34.178 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:34.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:34.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-261993005062 2026-02-13T19:11:34.180 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:34.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=261993005062 2026-02-13T19:11:34.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 261993005062' 2026-02-13T19:11:34.181 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 261993005062 2026-02-13T19:11:34.182 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:34.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 261993005062 -lt 261993005062 2026-02-13T19:11:34.410 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:34.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-304942678019 2026-02-13T19:11:34.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:34.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:34.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-304942678019 2026-02-13T19:11:34.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:34.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=304942678019 2026-02-13T19:11:34.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 304942678019' 2026-02-13T19:11:34.413 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 304942678019 2026-02-13T19:11:34.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:34.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 304942678019 -lt 304942678019 2026-02-13T19:11:34.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:34.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149713 2026-02-13T19:11:34.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:34.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:34.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149713 2026-02-13T19:11:34.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:34.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149713 2026-02-13T19:11:34.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149713' 2026-02-13T19:11:34.645 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149713 2026-02-13T19:11:34.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:34.872 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149711 -lt 111669149713 2026-02-13T19:11:34.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:35.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:35.874 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:36.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149713 -lt 111669149713 2026-02-13T19:11:36.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:36.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:36.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:36.416 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:36.417 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:36.417 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:36.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:36.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:36.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:36.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:36.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:36.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:36.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:36.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:11:36.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:11:37.286 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:11:37.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-13T19:11:37.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-0 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-0 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:11:37.297 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-0 td/test-erasure-eio/COPY 2026-02-13T19:11:37.314 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:11:37.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:11:37.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:11:37.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-13T19:11:37.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-94542-1-256 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:11:37.319 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-94542-1-256 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:11:37.320 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-94542-1-256 td/test-erasure-eio/ORIGINAL 2026-02-13T19:11:37.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-94542-1-256 td/test-erasure-eio 1 256 add 2026-02-13T19:11:37.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-256 2026-02-13T19:11:37.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:37.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:37.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-256 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:37.337 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:37.564 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:37.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-13T19:11:37.565 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:37.846 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:37.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-13T19:11:37.857 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:11:37.858 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:37.859 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:37.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:37.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:37.859 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:11:38.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:38.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:38.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:38.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:38.255 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:38.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:38.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:11:38.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:11:38.257 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:11:38.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:38.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:11:38.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:11:38.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:38.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:38.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:38.269 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:38.268+0000 7f8811e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:38.270 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:38.269+0000 7f8811e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:38.271 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:38.270+0000 7f8811e37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:38.403 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:38.402+0000 7f8811e37780 -1 Falling back to public interface 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:38.483 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:38.484 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:38.484 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:38.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:38.780 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:38.779+0000 7f8811e37780 -1 osd.1 75 log_to_monitors true 2026-02-13T19:11:39.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:39.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:39.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:39.709 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:39.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:39.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 80 up_thru 80 down_at 77 last_clean_interval [61,75) [v2:127.0.0.1:6810/2684604572,v1:127.0.0.1:6811/2684604572] [v2:127.0.0.1:6812/2684604572,v1:127.0.0.1:6813/2684604572] exists,up be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:39.934 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:39.935 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:39.935 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:39.935 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:39.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:39.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:40.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:40.022 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:40.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:40.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:40.023 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:40.023 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:40.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:40.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:40.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-13T19:11:40.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-13T19:11:40.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797' 2026-02-13T19:11:40.313 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:40.313 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:40.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383683 2026-02-13T19:11:40.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383683 2026-02-13T19:11:40.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-343597383683' 2026-02-13T19:11:40.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:40.379 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:40.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678022 2026-02-13T19:11:40.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678022 2026-02-13T19:11:40.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-343597383683 2-304942678022' 2026-02-13T19:11:40.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:40.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:40.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149716 2026-02-13T19:11:40.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149716 2026-02-13T19:11:40.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-343597383683 2-304942678022 3-111669149716' 2026-02-13T19:11:40.506 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:40.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803797 2026-02-13T19:11:40.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:40.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:40.508 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-13T19:11:40.508 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:40.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803797 2026-02-13T19:11:40.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-13T19:11:40.510 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803797 2026-02-13T19:11:40.510 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:40.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-13T19:11:40.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:41.738 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:41.739 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:41.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-13T19:11:41.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:42.965 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:11:42.965 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:43.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803797 2026-02-13T19:11:43.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:43.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-343597383683 2026-02-13T19:11:43.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:43.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:43.190 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-343597383683 2026-02-13T19:11:43.191 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:43.191 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=343597383683 2026-02-13T19:11:43.192 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 343597383683' 2026-02-13T19:11:43.192 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 343597383683 2026-02-13T19:11:43.192 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:43.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 343597383683 -lt 343597383683 2026-02-13T19:11:43.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:43.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-304942678022 2026-02-13T19:11:43.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:43.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:43.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-304942678022 2026-02-13T19:11:43.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:43.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=304942678022 2026-02-13T19:11:43.429 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 304942678022' 2026-02-13T19:11:43.430 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 304942678022 2026-02-13T19:11:43.430 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:43.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 304942678022 -lt 304942678022 2026-02-13T19:11:43.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:43.653 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149716 2026-02-13T19:11:43.654 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:43.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:43.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149716 2026-02-13T19:11:43.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:43.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149716 2026-02-13T19:11:43.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149716' 2026-02-13T19:11:43.656 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149716 2026-02-13T19:11:43.657 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:43.880 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149716 -lt 111669149716 2026-02-13T19:11:43.880 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:43.881 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:43.881 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:44.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:44.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:44.194 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:44.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:44.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:44.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:44.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:44.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:44.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:44.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:44.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-13T19:11:44.729 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:11:44.729 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:256 bytes copied, 2.0698e-05 s, 12.4 MB/s 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:44.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:11:44.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:44.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:44.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:44.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:44.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:44.834 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:44.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:44.987 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:44.988 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:44.989 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:11:44.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:11:44.991 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:11:44.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:44.991 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:11:44.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:11:44.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:44.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:44.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:45.001 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:44.999+0000 7f77e9e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:45.003 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:45.001+0000 7f77e9e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:45.004 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:45.002+0000 7f77e9e48780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:45.186 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:45.185+0000 7f77e9e48780 -1 Falling back to public interface 2026-02-13T19:11:45.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:11:45.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:45.218 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:45.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:45.558 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:45.556+0000 7f77e9e48780 -1 osd.1 81 log_to_monitors true 2026-02-13T19:11:46.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:46.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:46.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:46.442 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:46.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:46.443 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 85 up_thru 85 down_at 82 last_clean_interval [80,81) [v2:127.0.0.1:6810/3903444300,v1:127.0.0.1:6811/3903444300] [v2:127.0.0.1:6812/3903444300,v1:127.0.0.1:6813/3903444300] exists,up be1c470f-214b-41eb-a43d-2aa702a035ff 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:46.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:46.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:46.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:46.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:46.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:46.763 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:46.764 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:46.764 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:46.996 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:47.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803801 2026-02-13T19:11:47.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803801 2026-02-13T19:11:47.059 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801' 2026-02-13T19:11:47.060 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:47.060 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:47.122 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220163 2026-02-13T19:11:47.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220163 2026-02-13T19:11:47.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-365072220163' 2026-02-13T19:11:47.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:47.123 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:47.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=304942678025 2026-02-13T19:11:47.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 304942678025 2026-02-13T19:11:47.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-365072220163 2-304942678025' 2026-02-13T19:11:47.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:47.189 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:47.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149719 2026-02-13T19:11:47.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149719 2026-02-13T19:11:47.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-365072220163 2-304942678025 3-111669149719' 2026-02-13T19:11:47.254 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:47.254 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803801 2026-02-13T19:11:47.255 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:47.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:47.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803801 2026-02-13T19:11:47.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:47.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803801 2026-02-13T19:11:47.258 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803801' 2026-02-13T19:11:47.258 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803801 2026-02-13T19:11:47.258 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:47.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803799 -lt 25769803801 2026-02-13T19:11:47.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:48.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:48.483 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803801 2026-02-13T19:11:48.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:48.710 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-365072220163 2026-02-13T19:11:48.710 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:48.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:48.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-365072220163 2026-02-13T19:11:48.711 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:48.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220163 2026-02-13T19:11:48.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 365072220163' 2026-02-13T19:11:48.712 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 365072220163 2026-02-13T19:11:48.713 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:48.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220163 -lt 365072220163 2026-02-13T19:11:48.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:48.942 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-304942678025 2026-02-13T19:11:48.943 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:48.943 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:48.944 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-304942678025 2026-02-13T19:11:48.944 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:48.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=304942678025 2026-02-13T19:11:48.945 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 304942678025' 2026-02-13T19:11:48.945 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 304942678025 2026-02-13T19:11:48.945 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:49.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 304942678026 -lt 304942678025 2026-02-13T19:11:49.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:49.173 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149719 2026-02-13T19:11:49.174 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:49.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:49.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149719 2026-02-13T19:11:49.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:49.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149719 2026-02-13T19:11:49.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149719' 2026-02-13T19:11:49.176 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149719 2026-02-13T19:11:49.176 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:49.401 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149719 -lt 111669149719 2026-02-13T19:11:49.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:49.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:49.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:49.714 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:49.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:49.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:49.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:49.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:49.715 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:49.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:49.716 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:49.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:49.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:49.936 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:49.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:50.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:50.248 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:50.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:50.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:11:50.249 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:11:50.561 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:11:50.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-256 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:11:50.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-256 td/test-erasure-eio/COPY 2026-02-13T19:11:50.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:11:50.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:11:50.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-13T19:11:50.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-94542-1-256 td/test-erasure-eio 2 256 add 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-94542-1-256 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-13T19:11:50.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-94542-1-256 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-94542-1-256 2026-02-13T19:11:50.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:11:50.820 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-13T19:11:50.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-13T19:11:51.139 INFO:tasks.workunit.client.0.trial034.stderr:noout is set 2026-02-13T19:11:51.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-13T19:11:51.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:51.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:51.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:51.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:51.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:51.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:51.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-94542-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:51.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-13T19:11:51.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:51.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:51.541 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:51.542 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:51.543 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:start osd.2 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:51.544 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:11:51.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:11:51.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:51.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:51.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:51.555 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:51.553+0000 7f21567fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:51.556 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:51.555+0000 7f21567fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:51.557 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:51.555+0000 7f21567fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:51.689 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:51.687+0000 7f21567fb780 -1 Falling back to public interface 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:51.769 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:51.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:51.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:52.022 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:52.020+0000 7f21567fb780 -1 osd.2 89 log_to_monitors true 2026-02-13T19:11:52.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:52.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:52.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:52.994 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:52.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:52.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:53.235 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 up in weight 1 up_from 94 up_thru 82 down_at 91 last_clean_interval [71,89) [v2:127.0.0.1:6818/2093870787,v1:127.0.0.1:6819/2093870787] [v2:127.0.0.1:6820/2093870787,v1:127.0.0.1:6821/2093870787] exists,up 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:53.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:53.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:53.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:53.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:53.323 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:53.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:53.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:53.324 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:53.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:53.549 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:53.549 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:53.549 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:53.549 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:53.550 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:53.550 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-13T19:11:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-13T19:11:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804' 2026-02-13T19:11:53.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:53.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:53.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220166 2026-02-13T19:11:53.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220166 2026-02-13T19:11:53.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-365072220166' 2026-02-13T19:11:53.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:53.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:53.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=403726925827 2026-02-13T19:11:53.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 403726925827 2026-02-13T19:11:53.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-365072220166 2-403726925827' 2026-02-13T19:11:53.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:53.744 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:53.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149723 2026-02-13T19:11:53.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149723 2026-02-13T19:11:53.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-365072220166 2-403726925827 3-111669149723' 2026-02-13T19:11:53.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:53.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803804 2026-02-13T19:11:53.809 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:53.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:53.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-13T19:11:53.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803804 2026-02-13T19:11:53.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-13T19:11:53.811 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803804 2026-02-13T19:11:53.812 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:54.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-13T19:11:54.034 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:54.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-365072220166 2026-02-13T19:11:54.035 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:54.036 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:54.036 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-365072220166 2026-02-13T19:11:54.037 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:54.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220166 2026-02-13T19:11:54.039 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 365072220166' 2026-02-13T19:11:54.039 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 365072220166 2026-02-13T19:11:54.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:54.260 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220166 -lt 365072220166 2026-02-13T19:11:54.261 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:54.261 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-403726925827 2026-02-13T19:11:54.262 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:54.262 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:11:54.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-403726925827 2026-02-13T19:11:54.263 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:54.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=403726925827 2026-02-13T19:11:54.264 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 403726925827' 2026-02-13T19:11:54.264 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 403726925827 2026-02-13T19:11:54.265 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:11:54.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 403726925827 -lt 403726925827 2026-02-13T19:11:54.489 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:54.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149723 2026-02-13T19:11:54.490 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:54.490 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:11:54.491 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149723 2026-02-13T19:11:54.491 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:54.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149723 2026-02-13T19:11:54.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149723' 2026-02-13T19:11:54.492 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149723 2026-02-13T19:11:54.493 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:11:54.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149723 -lt 111669149723 2026-02-13T19:11:54.721 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:11:54.722 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:54.722 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:11:55.040 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:11:55.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:11:55.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:11:55.259 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:11:55.260 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:11:55.260 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:11:55.260 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:11:55.569 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:11:55.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:11:55.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:11:55.570 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-13T19:11:55.572 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records in 2026-02-13T19:11:55.572 INFO:tasks.workunit.client.0.trial034.stderr:1+0 records out 2026-02-13T19:11:55.572 INFO:tasks.workunit.client.0.trial034.stderr:256 bytes copied, 3.2079e-05 s, 8.0 MB/s 2026-02-13T19:11:55.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:55.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:11:55.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:11:55.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:11:55.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:11:55.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:11:55.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:11:55.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:55.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-94542-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:11:55.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:11:55.821 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:11:55.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:start osd.2 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:11:55.823 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:11:55.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:11:55.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:11:55.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:11:55.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:11:55.834 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:55.832+0000 7fe749814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:55.835 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:55.833+0000 7fe749814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:55.836 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:55.834+0000 7fe749814780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:11:55.995 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:55.993+0000 7fe749814780 -1 Falling back to public interface 2026-02-13T19:11:56.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:56.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:56.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:11:56.368 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:11:56.366+0000 7fe749814780 -1 osd.2 95 log_to_monitors true 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:11:57.269 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:11:57.500 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 up in weight 1 up_from 99 up_thru 82 down_at 96 last_clean_interval [94,95) [v2:127.0.0.1:6818/3415297505,v1:127.0.0.1:6819/3415297505] [v2:127.0.0.1:6820/3415297505,v1:127.0.0.1:6821/3415297505] exists,up 04183257-f772-4845-991f-3c44432cac4d 2026-02-13T19:11:57.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:11:57.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:11:57.501 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:11:57.502 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:11:57.502 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:11:57.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:11:57.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:11:57.502 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:11:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:11:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:11:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:11:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:11:57.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:11:57.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:11:57.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:11:57.817 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:11:57.817 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:11:57.817 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:11:57.817 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:11:57.818 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:57.818 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:11:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-13T19:11:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-13T19:11:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807' 2026-02-13T19:11:57.886 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:57.886 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:11:57.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220169 2026-02-13T19:11:57.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220169 2026-02-13T19:11:57.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-365072220169' 2026-02-13T19:11:57.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:57.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:11:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=425201762307 2026-02-13T19:11:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 425201762307 2026-02-13T19:11:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-365072220169 2-425201762307' 2026-02-13T19:11:58.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:11:58.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:11:58.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149726 2026-02-13T19:11:58.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149726 2026-02-13T19:11:58.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-365072220169 2-425201762307 3-111669149726' 2026-02-13T19:11:58.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:58.082 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803807 2026-02-13T19:11:58.082 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:58.082 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:11:58.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-13T19:11:58.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:58.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803807 2026-02-13T19:11:58.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-13T19:11:58.084 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803807 2026-02-13T19:11:58.085 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:11:58.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-13T19:11:58.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:11:58.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-365072220169 2026-02-13T19:11:58.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:11:58.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:11:58.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-365072220169 2026-02-13T19:11:58.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:11:58.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220169 2026-02-13T19:11:58.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 365072220169' 2026-02-13T19:11:58.329 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 365072220169 2026-02-13T19:11:58.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:58.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220167 -lt 365072220169 2026-02-13T19:11:58.553 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:11:59.554 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:11:59.554 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:11:59.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220167 -lt 365072220169 2026-02-13T19:11:59.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:12:00.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:12:00.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:12:01.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220169 -lt 365072220169 2026-02-13T19:12:01.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:01.015 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-425201762307 2026-02-13T19:12:01.015 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:01.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:12:01.016 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-425201762307 2026-02-13T19:12:01.017 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:01.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=425201762307 2026-02-13T19:12:01.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 425201762307' 2026-02-13T19:12:01.018 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 425201762307 2026-02-13T19:12:01.018 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:12:01.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 425201762307 -lt 425201762307 2026-02-13T19:12:01.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:01.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149726 2026-02-13T19:12:01.244 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:01.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:12:01.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149726 2026-02-13T19:12:01.246 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:01.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149726 2026-02-13T19:12:01.247 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149726' 2026-02-13T19:12:01.248 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149726 2026-02-13T19:12:01.248 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:12:01.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149726 -lt 111669149726 2026-02-13T19:12:01.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:12:01.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:01.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:01.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:12:01.788 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:12:01.788 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:12:01.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:12:01.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:12:01.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:12:01.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:12:01.789 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:12:02.012 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:12:02.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:12:02.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:02.013 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:02.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:12:02.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:12:02.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:12:02.330 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-13T19:12:02.331 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-13T19:12:02.643 INFO:tasks.workunit.client.0.trial034.stderr:noout is unset 2026-02-13T19:12:02.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-13T19:12:02.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-94542-1-256 2026-02-13T19:12:02.652 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:12:02.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:12:02.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-94542-1-256 2026-02-13T19:12:02.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:12:02.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:12:02.653 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-94542-1-256 td/test-erasure-eio/COPY 2026-02-13T19:12:02.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:12:02.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:12:02.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:02.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:12:02.673 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:02.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:12:02.910 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:12:02.918 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:12:03.194 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:12:03.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:03.203 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:03.311 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:03.312 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:03.312 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:03.313 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:03.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:03.315 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:03.315 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:03.316 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:03.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:03.317 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:03.317 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:03.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:03.318 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:03.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:03.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.335 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.336 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:03.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:03.337 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:03.338 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:03.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:03.339 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:03.339 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:03.340 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:03.340 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:03.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:03.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:03.341 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:03.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:03.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:03.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:03.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:03.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:03.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:03.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:03.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:03.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:03.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:03.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:03.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:12:03.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:12:03.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.348 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:12:03.349 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:12:03.351 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:12:03.368 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:03.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:12:03.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:12:03.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:12:03.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:03.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:03.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:12:03.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:03.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:12:03.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:12:03.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:03.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:03.438 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:12:03.439 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:03.440 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:03.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:12:03.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:12:03.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:12:03.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:12:03.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:12:03.481 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:12:03.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:12:03.482 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:03.580 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:03.581 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:03.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:03.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:12:03.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:12:03.692 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:12:03.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:04.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:12:04.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:12:04.739 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:04.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:03.385+0000 7f038943ac00 0 load: jerasure load: lrc 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:12:04.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:12:04.746 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:12:04.747 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:04.748 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:04.750 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:04.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:04.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:04.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:04.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:04.751 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:12:04.752 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:04.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fcf67b7d-50b7-470b-bf11-df1ca289f282 2026-02-13T19:12:04.752 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fcf67b7d-50b7-470b-bf11-df1ca289f282' 2026-02-13T19:12:04.752 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 fcf67b7d-50b7-470b-bf11-df1ca289f282 2026-02-13T19:12:04.752 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:04.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEd49pplBSLRAAkgY7gK9ecVSCwNy0ywaueA== 2026-02-13T19:12:04.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEd49pplBSLRAAkgY7gK9ecVSCwNy0ywaueA=="}' 2026-02-13T19:12:04.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fcf67b7d-50b7-470b-bf11-df1ca289f282 -i td/test-erasure-eio/0/new.json 2026-02-13T19:12:04.863 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:04.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:12:04.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEd49pplBSLRAAkgY7gK9ecVSCwNy0ywaueA== --osd-uuid fcf67b7d-50b7-470b-bf11-df1ca289f282 2026-02-13T19:12:04.887 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:04.886+0000 7f3d2fdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:04.888 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:04.887+0000 7f3d2fdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:04.889 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:04.888+0000 7f3d2fdfd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:04.889 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:04.888+0000 7f3d2fdfd780 -1 bdev(0x34cec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:04.889 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:04.889+0000 7f3d2fdfd780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:12:05.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:12:05.288 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:05.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:12:05.289 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:12:05.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:05.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:12:05.395 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:12:05.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:05.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:05.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:05.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:05.410 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:05.409+0000 7fb74ec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:05.411 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:05.410+0000 7fb74ec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:05.412 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:05.411+0000 7fb74ec43780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:05.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:12:05.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:05.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:12:05.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:05.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:05.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:05.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:05.493 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:05.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:05.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:05.588 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:05.598 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:05.598+0000 7fb74ec43780 -1 Falling back to public interface 2026-02-13T19:12:05.870 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:05.869+0000 7fb74ec43780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:12:06.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:06.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:06.590 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:06.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:06.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:06.821 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4243857232,v1:127.0.0.1:6803/4243857232] [v2:127.0.0.1:6804/4243857232,v1:127.0.0.1:6805/4243857232] exists,up fcf67b7d-50b7-470b-bf11-df1ca289f282 2026-02-13T19:12:06.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:06.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:06.825 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:06.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:06.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:06.828 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:06.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:12:06.829 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:06.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2534dc76-3cf9-415e-9571-9358f32ab2c8 2026-02-13T19:12:06.829 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 2534dc76-3cf9-415e-9571-9358f32ab2c8 2026-02-13T19:12:06.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2534dc76-3cf9-415e-9571-9358f32ab2c8' 2026-02-13T19:12:06.830 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:06.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGd49pmQq/MRAAWqjHJBTUgCTyxggrGCXEcQ== 2026-02-13T19:12:06.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGd49pmQq/MRAAWqjHJBTUgCTyxggrGCXEcQ=="}' 2026-02-13T19:12:06.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2534dc76-3cf9-415e-9571-9358f32ab2c8 -i td/test-erasure-eio/1/new.json 2026-02-13T19:12:07.065 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:07.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:12:07.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGd49pmQq/MRAAWqjHJBTUgCTyxggrGCXEcQ== --osd-uuid 2534dc76-3cf9-415e-9571-9358f32ab2c8 2026-02-13T19:12:07.089 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.088+0000 7fb7e5a2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.090 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.090+0000 7fb7e5a2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.091 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.091+0000 7fb7e5a2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.092 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.091+0000 7fb7e5a2c780 -1 bdev(0x2810c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:07.092 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.091+0000 7fb7e5a2c780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:12:07.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:12:07.422 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:07.423 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:12:07.423 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:12:07.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:07.743 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:12:07.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:12:07.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:07.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:07.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:07.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:07.759 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.758+0000 7fa803fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.760 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.759+0000 7fa803fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.761 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.760+0000 7fa803fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:07.919 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:07.919+0000 7fa803fe3780 -1 Falling back to public interface 2026-02-13T19:12:07.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:12:07.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:07.969 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:07.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:07.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:08.193 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:08.320 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:08.319+0000 7fa803fe3780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:12:08.761 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:08.760+0000 7fa7ff5f7640 -1 osd.1 0 waiting for initial osdmap 2026-02-13T19:12:09.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:09.195 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:09.195 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:09.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:09.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:09.196 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:09.421 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2691451735,v1:127.0.0.1:6811/2691451735] [v2:127.0.0.1:6812/2691451735,v1:127.0.0.1:6813/2691451735] exists,up 2534dc76-3cf9-415e-9571-9358f32ab2c8 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:12:09.425 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:09.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:09.427 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0d3b837a-5060-41f4-98ae-3edc900f0454 2026-02-13T19:12:09.428 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0d3b837a-5060-41f4-98ae-3edc900f0454' 2026-02-13T19:12:09.429 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 0d3b837a-5060-41f4-98ae-3edc900f0454 2026-02-13T19:12:09.429 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:09.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJd49pBYvaGRAAfUI3b2LmDeBcb8VWfSPY7A== 2026-02-13T19:12:09.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJd49pBYvaGRAAfUI3b2LmDeBcb8VWfSPY7A=="}' 2026-02-13T19:12:09.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0d3b837a-5060-41f4-98ae-3edc900f0454 -i td/test-erasure-eio/2/new.json 2026-02-13T19:12:09.663 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:12:09.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:12:09.671 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJd49pBYvaGRAAfUI3b2LmDeBcb8VWfSPY7A== --osd-uuid 0d3b837a-5060-41f4-98ae-3edc900f0454 2026-02-13T19:12:09.687 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:09.686+0000 7ff2f35d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:09.688 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:09.687+0000 7ff2f35d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:09.689 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:09.688+0000 7ff2f35d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:09.690 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:09.689+0000 7ff2f35d6780 -1 bdev(0x3c88c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:09.690 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:09.689+0000 7ff2f35d6780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:12:10.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:12:10.018 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:10.019 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:12:10.019 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:12:10.020 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:10.345 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:12:10.346 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:12:10.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:10.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:10.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:10.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:10.361 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:10.360+0000 7fca7f634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:10.363 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:10.362+0000 7fca7f634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:10.364 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:10.363+0000 7fca7f634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:10.492 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:10.491+0000 7fca7f634780 -1 Falling back to public interface 2026-02-13T19:12:10.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:10.576 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:10.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:10.577 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:10.772 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:10.771+0000 7fca7f634780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:12:10.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:11.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:11.811 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:11.812 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:11.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:11.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:11.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:12.037 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4174240701,v1:127.0.0.1:6819/4174240701] [v2:127.0.0.1:6820/4174240701,v1:127.0.0.1:6821/4174240701] exists,up 0d3b837a-5060-41f4-98ae-3edc900f0454 2026-02-13T19:12:12.041 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:12.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:12.043 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:12.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f' 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f 2026-02-13T19:12:12.045 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:12.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCMd49paxQIAxAA8+G/6yMuwfQyDxd//kozhA== 2026-02-13T19:12:12.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCMd49paxQIAxAA8+G/6yMuwfQyDxd//kozhA=="}' 2026-02-13T19:12:12.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f -i td/test-erasure-eio/3/new.json 2026-02-13T19:12:12.284 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:12:12.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:12:12.293 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMd49paxQIAxAA8+G/6yMuwfQyDxd//kozhA== --osd-uuid 8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f 2026-02-13T19:12:12.309 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.308+0000 7fe3561d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:12.310 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.309+0000 7fe3561d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:12.311 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.310+0000 7fe3561d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:12.311 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.310+0000 7fe3561d6780 -1 bdev(0x2cbec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:12.311 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.310+0000 7fe3561d6780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:12:12.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:12:12.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:12.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:12:12.635 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:12:12.635 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:12.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:12:12.956 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:12:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:12.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:12.971 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.970+0000 7f1f59d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:12.972 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.971+0000 7f1f59d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:12.973 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:12.972+0000 7f1f59d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:13.154 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:13.153+0000 7f1f59d3d780 -1 Falling back to public interface 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:13.183 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:13.184 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:13.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:13.184 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:13.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:13.509 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:13.508+0000 7f1f59d3d780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:12:14.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:14.417 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:14.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:14.418 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:14.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:14.418 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:14.641 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2123009127,v1:127.0.0.1:6827/2123009127] [v2:127.0.0.1:6828/2123009127,v1:127.0.0.1:6829/2123009127] exists,up 8f6b13f7-a7c7-478d-bf5a-02cb10ce7e0f 2026-02-13T19:12:14.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:14.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:14.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:12:14.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:12:14.688 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:14.691 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:12:14.694 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:05.600+0000 7fb74ec43780 0 load: jerasure load: lrc 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:12:14.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:12:14.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:14.980 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:15.355 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:12:15.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:16.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:12:16.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:12:16.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:12:16.365 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:12:16.366 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:12:16.366 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:12:16.366 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:12:16.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:12:16.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:12:16.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:12:16.454 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:16.680 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:12:16.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:12:16.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:12:16.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:12:16.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:16.743 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:12:16.809 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-13T19:12:16.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-13T19:12:16.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557' 2026-02-13T19:12:16.810 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:16.810 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:12:16.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-13T19:12:16.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-13T19:12:16.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-73014444035' 2026-02-13T19:12:16.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:16.874 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:12:16.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215107 2026-02-13T19:12:16.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215107 2026-02-13T19:12:16.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-73014444035 3-103079215107' 2026-02-13T19:12:16.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:16.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:12:16.937 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:16.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:12:16.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:12:16.938 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:16.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:12:16.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:12:16.940 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:12:16.940 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:17.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:12:17.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:12:18.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:12:18.166 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:18.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:12:18.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:18.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607557 2026-02-13T19:12:18.395 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:18.396 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:12:18.396 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-13T19:12:18.397 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:18.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607557 2026-02-13T19:12:18.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-13T19:12:18.398 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607557 2026-02-13T19:12:18.398 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:12:18.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-13T19:12:18.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:18.633 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444035 2026-02-13T19:12:18.633 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:18.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:12:18.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-13T19:12:18.635 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:18.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444035 2026-02-13T19:12:18.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-13T19:12:18.636 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 73014444035 2026-02-13T19:12:18.636 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:12:18.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444036 -lt 73014444035 2026-02-13T19:12:18.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:18.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215107 2026-02-13T19:12:18.865 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:18.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:12:18.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215107 2026-02-13T19:12:18.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:18.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215107 2026-02-13T19:12:18.867 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215107' 2026-02-13T19:12:18.868 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 103079215107 2026-02-13T19:12:18.868 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:12:19.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215107 -lt 103079215107 2026-02-13T19:12:19.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:12:19.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:19.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:19.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:12:19.424 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:12:19.425 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:12:19.647 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:12:19.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:12:19.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:19.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-13T19:12:19.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-94542-0 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-94542-0 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:12:19.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-94542-0 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:12:19.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-94542-0 td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-0 td/test-erasure-eio 0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-0 2026-02-13T19:12:19.978 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:12:20.204 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:12:20.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:12:20.206 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:12:20.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:12:20.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:12:20.206 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:12:20.207 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:20.207 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:20.207 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:20.207 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:12:20.207 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:12:20.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:12:20.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:12:20.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:12:20.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:12:20.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:12:20.256 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:12:20.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-94542-0 0 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-0 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-0 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:12:20.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-94542-0 td/test-erasure-eio/COPY 2026-02-13T19:12:20.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:12:20.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:12:20.327 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-13T19:12:20.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-0 td/test-erasure-eio 1 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:12:20.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-0 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-0 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-0 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-0 2026-02-13T19:12:20.329 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:12:20.565 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:12:20.565 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:20.565 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:12:20.566 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:12:20.567 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:20.568 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:12:20.569 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:12:20.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:12:20.616 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:20.617 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:20.618 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:12:20.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:12:20.618 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-94542-0 1 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-0 fail 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-0 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-13T19:12:20.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-13T19:12:20.666 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-94542-0 td/test-erasure-eio/COPY 2026-02-13T19:12:20.681 INFO:tasks.workunit.client.0.trial034.stderr:error getting pool-jerasure/obj-eio-94542-0: (5) Input/output error 2026-02-13T19:12:20.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-13T19:12:20.682 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:20.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:12:20.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:20.683 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:12:20.954 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:12:20.962 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:12:21.237 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:12:21.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:12:21.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:21.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:21.244 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:21.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:21.245 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:21.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:21.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:21.245 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:21.353 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:21.354 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:21.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:21.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:21.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:21.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:21.357 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:21.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:21.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:21.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:21.359 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:21.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:21.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:21.361 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:21.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:21.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.369 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:21.371 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:21.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:21.373 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:21.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:21.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:21.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:21.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:21.376 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:21.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:21.376 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:21.377 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:21.377 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:21.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:21.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:21.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:21.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:21.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:21.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:21.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:21.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:12:21.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:12:21.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:12:21.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:21.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:12:21.426 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:12:21.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:12:21.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:21.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:21.426 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.427 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.428 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:21.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:21.428 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:21.470 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:21.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:12:21.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:12:21.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:12:21.613 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:21.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:21.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:21.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:21.614 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:21.614 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:21.615 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:21.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:21.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:12:21.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:12:21.727 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:12:21.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:22.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:12:22.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:22.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:12:22.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:12:22.776 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:22.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:21.418+0000 7f824062fc00 0 load: jerasure load: lrc 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:12:22.782 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:12:22.783 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:12:22.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:22.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:12:22.784 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:22.785 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:12:22.786 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d22fa6ea-f2aa-413c-9b8e-f3094707b47c 2026-02-13T19:12:22.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d22fa6ea-f2aa-413c-9b8e-f3094707b47c' 2026-02-13T19:12:22.787 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 d22fa6ea-f2aa-413c-9b8e-f3094707b47c 2026-02-13T19:12:22.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:22.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWd49pjV6KLxAAda3jEp/bM8+wdjSbLUQJkg== 2026-02-13T19:12:22.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWd49pjV6KLxAAda3jEp/bM8+wdjSbLUQJkg=="}' 2026-02-13T19:12:22.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d22fa6ea-f2aa-413c-9b8e-f3094707b47c -i td/test-erasure-eio/0/new.json 2026-02-13T19:12:22.898 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:22.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:12:22.906 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWd49pjV6KLxAAda3jEp/bM8+wdjSbLUQJkg== --osd-uuid d22fa6ea-f2aa-413c-9b8e-f3094707b47c 2026-02-13T19:12:22.922 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:22.921+0000 7fbe49c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:22.923 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:22.923+0000 7fbe49c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:22.924 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:22.924+0000 7fbe49c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:22.925 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:22.924+0000 7fbe49c2d780 -1 bdev(0x300ec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:22.925 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:22.924+0000 7fbe49c2d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:12:23.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:12:23.324 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:23.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:12:23.325 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:12:23.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:23.431 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:12:23.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:12:23.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:23.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:23.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:23.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:23.446 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:23.446+0000 7fe5e4c5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:23.448 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:23.447+0000 7fe5e4c5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:23.449 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:23.448+0000 7fe5e4c5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:23.531 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:23.532 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:23.532 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:23.532 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:23.630 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:23.657 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:23.657+0000 7fe5e4c5b780 -1 Falling back to public interface 2026-02-13T19:12:23.925 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:23.924+0000 7fe5e4c5b780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:12:24.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:24.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:24.632 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:24.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:24.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:24.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:24.859 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1085544615,v1:127.0.0.1:6803/1085544615] [v2:127.0.0.1:6804/1085544615,v1:127.0.0.1:6805/1085544615] exists,up d22fa6ea-f2aa-413c-9b8e-f3094707b47c 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:12:24.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:24.864 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:24.865 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:24.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:24.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:12:24.866 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:24.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c591740f-bd02-48e4-a710-4a5b4f20bb10 2026-02-13T19:12:24.866 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c591740f-bd02-48e4-a710-4a5b4f20bb10' 2026-02-13T19:12:24.867 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 c591740f-bd02-48e4-a710-4a5b4f20bb10 2026-02-13T19:12:24.867 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:24.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCYd49phNMBNBAAE81CDwQJsPZtZpz3PQyfBg== 2026-02-13T19:12:24.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCYd49phNMBNBAAE81CDwQJsPZtZpz3PQyfBg=="}' 2026-02-13T19:12:24.873 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c591740f-bd02-48e4-a710-4a5b4f20bb10 -i td/test-erasure-eio/1/new.json 2026-02-13T19:12:25.105 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:25.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:12:25.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCYd49phNMBNBAAE81CDwQJsPZtZpz3PQyfBg== --osd-uuid c591740f-bd02-48e4-a710-4a5b4f20bb10 2026-02-13T19:12:25.129 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.129+0000 7f9213c55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.131 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.130+0000 7f9213c55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.131+0000 7f9213c55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.131+0000 7f9213c55780 -1 bdev(0x3b96c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:25.132 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.131+0000 7f9213c55780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:12:25.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:12:25.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:25.458 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:12:25.458 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:12:25.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:25.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:25.797 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.796+0000 7f5f9a42b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.798 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.798+0000 7f5f9a42b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.799 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.798+0000 7f5f9a42b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:25.966 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:25.965+0000 7f5f9a42b780 -1 Falling back to public interface 2026-02-13T19:12:26.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:12:26.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:26.007 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:26.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:26.008 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:26.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:26.345 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:26.344+0000 7f5f9a42b780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:12:27.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:27.234 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:27.234 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:27.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:27.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:27.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:27.459 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3168869915,v1:127.0.0.1:6811/3168869915] [v2:127.0.0.1:6812/3168869915,v1:127.0.0.1:6813/3168869915] exists,up c591740f-bd02-48e4-a710-4a5b4f20bb10 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:27.463 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:27.464 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:27.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:27.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b7cdbb26-066a-416f-a592-90a0a36416a8 2026-02-13T19:12:27.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b7cdbb26-066a-416f-a592-90a0a36416a8' 2026-02-13T19:12:27.467 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 b7cdbb26-066a-416f-a592-90a0a36416a8 2026-02-13T19:12:27.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:27.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCbd49pY+8mHBAAwlUjE719lGXElruVcIH/zw== 2026-02-13T19:12:27.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCbd49pY+8mHBAAwlUjE719lGXElruVcIH/zw=="}' 2026-02-13T19:12:27.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b7cdbb26-066a-416f-a592-90a0a36416a8 -i td/test-erasure-eio/2/new.json 2026-02-13T19:12:27.702 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:12:27.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:12:27.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbd49pY+8mHBAAwlUjE719lGXElruVcIH/zw== --osd-uuid b7cdbb26-066a-416f-a592-90a0a36416a8 2026-02-13T19:12:27.726 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:27.726+0000 7fb481a31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:27.728 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:27.727+0000 7fb481a31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:27.728 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:27.728+0000 7fb481a31780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:27.729 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:27.728+0000 7fb481a31780 -1 bdev(0x312cc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:27.729 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:27.728+0000 7fb481a31780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:12:28.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:12:28.042 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:28.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:12:28.043 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:12:28.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:28.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:12:28.363 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:12:28.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:28.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:28.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:28.364 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:28.378 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:28.377+0000 7f2158032780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:28.379 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:28.378+0000 7f2158032780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:28.380 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:28.379+0000 7f2158032780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:28.504 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:28.503+0000 7f2158032780 -1 Falling back to public interface 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:28.589 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:28.590 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:28.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:28.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:28.591 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:28.792 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:28.792+0000 7f2158032780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:12:28.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:29.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:29.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:29.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:29.823 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:29.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:29.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:30.050 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3240088950,v1:127.0.0.1:6819/3240088950] [v2:127.0.0.1:6820/3240088950,v1:127.0.0.1:6821/3240088950] exists,up b7cdbb26-066a-416f-a592-90a0a36416a8 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:12:30.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:30.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:30.057 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0463fb97-ace1-4956-b94e-246dba9d2765 2026-02-13T19:12:30.058 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 0463fb97-ace1-4956-b94e-246dba9d2765' 2026-02-13T19:12:30.059 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 0463fb97-ace1-4956-b94e-246dba9d2765 2026-02-13T19:12:30.059 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:30.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCed49pmfXJAxAA5089AqgCCUK1cVOqpwUx8g== 2026-02-13T19:12:30.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCed49pmfXJAxAA5089AqgCCUK1cVOqpwUx8g=="}' 2026-02-13T19:12:30.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0463fb97-ace1-4956-b94e-246dba9d2765 -i td/test-erasure-eio/3/new.json 2026-02-13T19:12:30.299 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:12:30.307 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:12:30.308 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCed49pmfXJAxAA5089AqgCCUK1cVOqpwUx8g== --osd-uuid 0463fb97-ace1-4956-b94e-246dba9d2765 2026-02-13T19:12:30.324 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:30.323+0000 7f5c32c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:30.325 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:30.324+0000 7f5c32c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:30.326 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:30.325+0000 7f5c32c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:30.326 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:30.325+0000 7f5c32c2d780 -1 bdev(0x43bcc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:30.327 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:30.325+0000 7f5c32c2d780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:12:30.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:12:30.669 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:30.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:12:30.670 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:12:30.670 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:30.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:12:30.995 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:12:30.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:30.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:30.996 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:30.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:31.010 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:31.010+0000 7fca40c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:31.012 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:31.011+0000 7fca40c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:31.013 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:31.012+0000 7fca40c2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:31.146 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:31.146+0000 7fca40c2d780 -1 Falling back to public interface 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:31.224 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:31.225 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:31.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:31.225 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:31.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:31.536 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:31.535+0000 7fca40c2d780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:12:32.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:32.449 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:32.450 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:32.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:32.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:32.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:32.672 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/483290056,v1:127.0.0.1:6827/483290056] [v2:127.0.0.1:6828/483290056,v1:127.0.0.1:6829/483290056] exists,up 0463fb97-ace1-4956-b94e-246dba9d2765 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:32.674 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:32.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:12:32.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:12:32.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:12:32.719 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:32.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:12:32.725 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:23.659+0000 7fe5e4c5b780 0 load: jerasure load: lrc 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:12:32.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:12:32.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:12:33.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:33.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:33.391 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:12:33.398 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:34.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:12:34.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:12:34.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:12:34.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:12:34.401 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:12:34.401 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:12:34.402 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:12:34.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:12:34.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:12:34.402 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:12:34.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:12:34.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:12:34.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:12:34.487 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:12:34.488 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:12:34.488 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:12:34.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:12:34.723 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:34.723 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:12:34.723 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:12:34.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:12:34.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:34.724 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:12:34.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-13T19:12:34.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-13T19:12:34.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-13T19:12:34.787 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:34.787 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:12:34.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-13T19:12:34.852 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-13T19:12:34.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557' 2026-02-13T19:12:34.853 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:34.853 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:12:34.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-13T19:12:34.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-13T19:12:34.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-77309411332' 2026-02-13T19:12:34.917 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:34.917 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:12:34.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-13T19:12:34.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-13T19:12:34.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-77309411332 3-107374182403' 2026-02-13T19:12:34.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:34.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-13T19:12:34.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:34.983 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:12:34.983 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-13T19:12:34.983 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:34.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-13T19:12:34.984 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-13T19:12:34.985 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803781 2026-02-13T19:12:34.985 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:35.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-13T19:12:35.211 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:12:36.212 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:12:36.213 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:36.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-13T19:12:36.445 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:36.445 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607557 2026-02-13T19:12:36.445 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:36.446 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:12:36.447 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-13T19:12:36.447 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:36.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607557 2026-02-13T19:12:36.448 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-13T19:12:36.449 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607557 2026-02-13T19:12:36.449 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:12:36.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-13T19:12:36.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:36.676 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-13T19:12:36.676 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:36.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:12:36.677 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-13T19:12:36.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:36.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-13T19:12:36.679 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411332 2026-02-13T19:12:36.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-13T19:12:36.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:12:36.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-13T19:12:36.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:36.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182403 2026-02-13T19:12:36.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:36.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:12:36.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-13T19:12:36.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:36.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182403 2026-02-13T19:12:36.911 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-13T19:12:36.911 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 107374182403 2026-02-13T19:12:36.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:12:37.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-13T19:12:37.136 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:12:37.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:37.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:37.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:12:37.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:12:37.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:12:37.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:12:37.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:12:37.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:12:37.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:12:37.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:12:37.678 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:12:37.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:12:37.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:37.679 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:37.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-13T19:12:37.993 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-94542-1 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-94542-1 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-94542-1 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:12:37.994 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:37.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:12:37.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:37.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:12:37.995 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-94542-1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-1 td/test-erasure-eio 1 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:12:38.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-1 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-1 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.011 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:12:38.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:12:38.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-13T19:12:38.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:12:38.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-13T19:12:38.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:12:38.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-13T19:12:38.240 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:38.241 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:12:38.242 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:12:38.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:12:38.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:12:38.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:12:38.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-13T19:12:38.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:12:38.291 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:12:38.291 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:38.292 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:38.292 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:38.292 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:12:38.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:12:38.292 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-94542-1 1 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-1 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:12:38.341 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:12:38.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-94542-1 td/test-erasure-eio/COPY 2026-02-13T19:12:38.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:12:38.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:12:38.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-13T19:12:38.360 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-13T19:12:38.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-13T19:12:38.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-1 td/test-erasure-eio 2 2026-02-13T19:12:38.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:12:38.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:12:38.362 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-1 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=2 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-1 2026-02-13T19:12:38.363 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.364 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:12:38.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:12:38.591 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:38.591 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=2 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:12:38.592 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/2/type 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:12:38.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.2.asok 2026-02-13T19:12:38.595 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:12:38.641 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:12:38.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:12:38.642 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.2 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:38.643 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:38.644 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.2.asok 2026-02-13T19:12:38.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:12:38.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-94542-1 2 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-1 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-1 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:12:38.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-94542-1 td/test-erasure-eio/COPY 2026-02-13T19:12:38.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:12:38.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:12:38.711 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:38.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:12:38.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:38.712 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:12:38.984 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:12:38.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:12:39.266 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:12:39.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:12:39.274 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:39.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:39.275 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:39.276 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:39.276 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:39.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:39.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:39.276 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:39.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:39.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:39.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:39.386 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:39.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:39.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:39.388 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:39.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:39.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:39.389 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:39.389 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:39.390 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:39.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:39.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:39.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:39.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.400 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:12:39.402 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:39.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:39.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:12:39.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:12:39.405 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:12:39.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:12:39.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:12:39.406 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:12:39.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:39.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:12:39.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:12:39.408 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:12:39.408 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:12:39.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:12:39.409 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:12:39.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:12:39.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.410 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:12:39.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:12:39.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:12:39.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:12:39.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:12:39.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.413 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:12:39.414 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:12:39.415 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:12:39.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:12:39.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:12:39.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:12:39.416 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.433 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.434 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:39.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:12:39.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:12:39.458 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:12:39.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:39.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:39.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:12:39.459 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.460 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.461 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:39.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:39.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:12:39.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:12:39.504 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:12:39.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:12:39.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:12:39.505 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.506 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.507 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:39.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:12:39.507 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:12:39.548 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:12:39.647 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:39.648 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:39.649 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:12:39.664 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:12:39.665 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:12:39.760 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:12:39.766 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:40.769 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:12:40.769 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:12:40.769 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:12:40.769 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:40.769 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:40.770 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:40.770 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:12:40.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:12:40.770 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:12:40.808 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:40.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:12:40.813 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:39.448+0000 7f0c34e14c00 0 load: jerasure load: lrc 2026-02-13T19:12:40.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-13T19:12:40.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-13T19:12:40.814 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-13T19:12:40.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:12:40.815 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:12:40.815 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:12:40.815 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:12:40.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:40.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:12:40.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:40.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:40.819 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:40.820 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:40.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b20454eb-3283-46b2-b881-b35cf1b4876e 2026-02-13T19:12:40.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b20454eb-3283-46b2-b881-b35cf1b4876e' 2026-02-13T19:12:40.823 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 b20454eb-3283-46b2-b881-b35cf1b4876e 2026-02-13T19:12:40.823 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:40.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCod49pJxNiMRAAagmL2gFNneAsmBtyD7IiZQ== 2026-02-13T19:12:40.829 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCod49pJxNiMRAAagmL2gFNneAsmBtyD7IiZQ=="}' 2026-02-13T19:12:40.830 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b20454eb-3283-46b2-b881-b35cf1b4876e -i td/test-erasure-eio/0/new.json 2026-02-13T19:12:40.930 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:40.936 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:12:40.937 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCod49pJxNiMRAAagmL2gFNneAsmBtyD7IiZQ== --osd-uuid b20454eb-3283-46b2-b881-b35cf1b4876e 2026-02-13T19:12:40.953 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:40.951+0000 7eff0843a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:40.954 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:40.952+0000 7eff0843a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:40.955 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:40.953+0000 7eff0843a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:40.955 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:40.953+0000 7eff0843a780 -1 bdev(0x41bcc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:40.956 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:40.954+0000 7eff0843a780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:12:41.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:12:41.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:41.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:12:41.395 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:12:41.395 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:41.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:12:41.500 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:12:41.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:41.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:41.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:41.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:41.515 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:41.513+0000 7fd403c0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:41.516 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:41.515+0000 7fd403c0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:41.517 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:41.515+0000 7fd403c0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:41.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:41.596 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:41.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:41.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:41.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:41.697 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:41.715 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:41.714+0000 7fd403c0b780 -1 Falling back to public interface 2026-02-13T19:12:41.991 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:41.989+0000 7fd403c0b780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:12:42.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:42.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:42.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:42.699 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:42.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:42.700 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:12:42.924 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2724661595,v1:127.0.0.1:6803/2724661595] [v2:127.0.0.1:6804/2724661595,v1:127.0.0.1:6805/2724661595] exists,up b20454eb-3283-46b2-b881-b35cf1b4876e 2026-02-13T19:12:42.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:42.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:42.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:42.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:42.930 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:42.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:12:42.932 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:42.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=219b7629-9902-4cba-b64f-c93530e91530 2026-02-13T19:12:42.932 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 219b7629-9902-4cba-b64f-c93530e91530 2026-02-13T19:12:42.932 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 219b7629-9902-4cba-b64f-c93530e91530' 2026-02-13T19:12:42.933 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:42.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqd49pJ43oNxAAl364t4YNesRLf2KFShLBuQ== 2026-02-13T19:12:42.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqd49pJ43oNxAAl364t4YNesRLf2KFShLBuQ=="}' 2026-02-13T19:12:42.939 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 219b7629-9902-4cba-b64f-c93530e91530 -i td/test-erasure-eio/1/new.json 2026-02-13T19:12:43.172 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:43.179 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:12:43.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCqd49pJ43oNxAAl364t4YNesRLf2KFShLBuQ== --osd-uuid 219b7629-9902-4cba-b64f-c93530e91530 2026-02-13T19:12:43.195 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.193+0000 7f25abe13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.196 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.194+0000 7f25abe13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.197 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.195+0000 7f25abe13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.198 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.196+0000 7f25abe13780 -1 bdev(0x2560c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:43.198 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.196+0000 7f25abe13780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:12:43.522 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:12:43.522 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:43.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:12:43.523 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:12:43.523 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:43.838 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:12:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:12:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:43.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:43.852 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.850+0000 7f115fa60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.853 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.851+0000 7f115fa60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.854 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.852+0000 7f115fa60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:43.982 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:43.980+0000 7f115fa60780 -1 Falling back to public interface 2026-02-13T19:12:44.064 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:44.065 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:44.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:44.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:44.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:44.327 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:44.325+0000 7f115fa60780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:12:45.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:45.301 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:45.301 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:45.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:45.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:45.302 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:45.529 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2276512119,v1:127.0.0.1:6811/2276512119] [v2:127.0.0.1:6812/2276512119,v1:127.0.0.1:6813/2276512119] exists,up 219b7629-9902-4cba-b64f-c93530e91530 2026-02-13T19:12:45.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:45.533 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:45.534 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:45.535 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:45.536 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:45.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:12:45.537 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:45.537 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90aef19f-e528-4015-a943-76b8b45371d7 2026-02-13T19:12:45.537 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 90aef19f-e528-4015-a943-76b8b45371d7 2026-02-13T19:12:45.538 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 90aef19f-e528-4015-a943-76b8b45371d7' 2026-02-13T19:12:45.538 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:45.544 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCtd49pK3BkIBAAaNieeTfNfoW7OWVn+Gbx8Q== 2026-02-13T19:12:45.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCtd49pK3BkIBAAaNieeTfNfoW7OWVn+Gbx8Q=="}' 2026-02-13T19:12:45.545 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90aef19f-e528-4015-a943-76b8b45371d7 -i td/test-erasure-eio/2/new.json 2026-02-13T19:12:45.774 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:12:45.782 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:12:45.783 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCtd49pK3BkIBAAaNieeTfNfoW7OWVn+Gbx8Q== --osd-uuid 90aef19f-e528-4015-a943-76b8b45371d7 2026-02-13T19:12:45.798 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:45.797+0000 7fa9f9225780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:45.800 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:45.798+0000 7fa9f9225780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:45.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:45.799+0000 7fa9f9225780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:45.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:45.799+0000 7fa9f9225780 -1 bdev(0x2580c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:45.801 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:45.799+0000 7fa9f9225780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:12:46.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:12:46.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:46.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:12:46.129 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:12:46.129 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:46.452 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:12:46.452 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:12:46.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:46.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:46.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:46.453 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:46.467 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:46.465+0000 7f8c28211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:46.468 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:46.466+0000 7f8c28211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:46.469 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:46.467+0000 7f8c28211780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:46.594 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:46.592+0000 7f8c28211780 -1 Falling back to public interface 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:46.679 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:46.680 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:46.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:46.680 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:46.901 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:46.900+0000 7f8c28211780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:12:46.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:47.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:47.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:47.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:47.907 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:47.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:47.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:12:48.137 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/246315529,v1:127.0.0.1:6819/246315529] [v2:127.0.0.1:6820/246315529,v1:127.0.0.1:6821/246315529] exists,up 90aef19f-e528-4015-a943-76b8b45371d7 2026-02-13T19:12:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:48.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:12:48.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:48.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:12:48.144 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=25e5603c-f9c0-4c02-b12a-3dcebd88775b 2026-02-13T19:12:48.145 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 25e5603c-f9c0-4c02-b12a-3dcebd88775b 2026-02-13T19:12:48.145 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 25e5603c-f9c0-4c02-b12a-3dcebd88775b' 2026-02-13T19:12:48.145 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:12:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCwd49p91f7CBAAf7/1W298IvB7/0R1+cRXag== 2026-02-13T19:12:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCwd49p91f7CBAAf7/1W298IvB7/0R1+cRXag=="}' 2026-02-13T19:12:48.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 25e5603c-f9c0-4c02-b12a-3dcebd88775b -i td/test-erasure-eio/3/new.json 2026-02-13T19:12:48.378 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:12:48.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:12:48.387 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCwd49p91f7CBAAf7/1W298IvB7/0R1+cRXag== --osd-uuid 25e5603c-f9c0-4c02-b12a-3dcebd88775b 2026-02-13T19:12:48.402 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:48.401+0000 7fbd92450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:48.404 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:48.402+0000 7fbd92450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:48.405 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:48.403+0000 7fbd92450780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:48.405 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:48.403+0000 7fbd92450780 -1 bdev(0x2b1ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:12:48.406 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:48.404+0000 7fbd92450780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:12:48.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:12:48.715 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:12:48.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:12:48.716 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:12:48.716 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:12:49.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:12:49.054 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:12:49.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:49.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:12:49.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:12:49.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:12:49.069 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:49.067+0000 7f22a5869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:49.070 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:49.069+0000 7f22a5869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:49.071 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:49.069+0000 7f22a5869780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:49.195 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:49.193+0000 7f22a5869780 -1 Falling back to public interface 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:49.283 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:49.284 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:12:49.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:49.284 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:49.502 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:49.500+0000 7f22a5869780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:12:49.508 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:50.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:50.510 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:50.510 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:12:50.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:50.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:50.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:12:50.739 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/342597265,v1:127.0.0.1:6827/342597265] [v2:127.0.0.1:6828/342597265,v1:127.0.0.1:6829/342597265] exists,up 25e5603c-f9c0-4c02-b12a-3dcebd88775b 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:12:50.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:12:50.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:12:50.786 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:12:50.789 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:12:50.791 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:12:41.716+0000 7fd403c0b780 0 load: jerasure load: lrc 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:12:50.792 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:12:50.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:12:50.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:12:50.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:12:51.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:51.078 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:12:51.449 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:12:51.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:12:52.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:12:52.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:12:52.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:12:52.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:12:52.461 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:12:52.552 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:52.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:12:52.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:12:52.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:12:52.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:12:52.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:52.845 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:12:52.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-13T19:12:52.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-13T19:12:52.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-13T19:12:52.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:52.910 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:12:52.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-13T19:12:52.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-13T19:12:52.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-13T19:12:52.972 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:52.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:12:53.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-13T19:12:53.037 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-13T19:12:53.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-111669149699' 2026-02-13T19:12:53.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:53.038 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:12:53.038 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:53.038 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:12:53.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:12:53.039 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:53.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:12:53.040 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:12:53.040 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:12:53.041 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:53.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:12:53.265 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:12:54.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:12:54.267 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:54.492 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:12:54.493 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:54.493 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-13T19:12:54.493 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:54.494 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:12:54.494 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-13T19:12:54.495 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:54.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-13T19:12:54.496 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-13T19:12:54.496 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 51539607556 2026-02-13T19:12:54.497 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:12:54.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-13T19:12:54.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:54.728 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-13T19:12:54.729 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:54.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:12:54.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-13T19:12:54.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-13T19:12:54.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-13T19:12:54.731 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411331 2026-02-13T19:12:54.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:12:54.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-13T19:12:54.956 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:54.957 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-13T19:12:54.957 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:54.957 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:12:54.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-13T19:12:54.958 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:54.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-13T19:12:54.959 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-13T19:12:54.960 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149699 2026-02-13T19:12:54.960 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:12:55.186 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-13T19:12:55.187 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:12:55.187 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:55.188 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:12:55.514 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:12:55.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:12:55.515 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:12:55.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:12:55.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:12:55.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:12:55.735 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:12:56.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:12:56.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:12:56.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:12:56.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-13T19:12:56.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-94542-1 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-94542-1 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:12:56.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-94542-1 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:12:56.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-94542-1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:12:56.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-94542-1 td/test-erasure-eio 1 2026-02-13T19:12:56.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-13T19:12:56.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-13T19:12:56.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-94542-1 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-94542-1 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:12:56.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-94542-1 2026-02-13T19:12:56.070 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-94542-1 2026-02-13T19:12:56.070 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:12:56.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-94542-1 remove 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:12:56.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-94542-1 remove 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:12:56.300 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:12:56.606 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-94542-1 remove 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-13T19:12:56.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-94542-1 remove 2026-02-13T19:12:56.700 INFO:tasks.workunit.client.0.trial034.stdout:remove 1#2:9c5f4564:::obj-remove-94542-1:head# 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:12:56.740 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:56.741 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-13T19:12:56.742 INFO:tasks.workunit.client.0.trial034.stderr:start osd.1 2026-02-13T19:12:56.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:12:56.743 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-13T19:12:56.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-13T19:12:56.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:12:56.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:12:56.744 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:12:56.753 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:56.752+0000 7f7dbf1fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:56.754 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:56.753+0000 7f7dbf1fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:56.755 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:56.754+0000 7f7dbf1fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:12:56.910 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:56.910+0000 7f7dbf1fd780 -1 Falling back to public interface 2026-02-13T19:12:56.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-13T19:12:56.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:12:56.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:12:56.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:12:56.963 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:12:56.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:56.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:12:56.964 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:12:56.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:56.964 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:57.197 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:12:57.196+0000 7f7dbf1fd780 -1 osd.1 34 log_to_monitors true 2026-02-13T19:12:57.202 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:12:58.204 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:12:58.431 INFO:tasks.workunit.client.0.trial034.stderr:osd.1 up in weight 1 up_from 38 up_thru 38 down_at 35 last_clean_interval [12,34) [v2:127.0.0.1:6810/3895453331,v1:127.0.0.1:6811/3895453331] [v2:127.0.0.1:6812/3895453331,v1:127.0.0.1:6813/3895453331] exists,up 219b7629-9902-4cba-b64f-c93530e91530 2026-02-13T19:12:58.431 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:12:58.432 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:12:58.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:12:58.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:12:58.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:12:58.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:12:58.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:12:58.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:12:58.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:12:58.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:12:58.519 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:12:58.745 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:12:58.745 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:12:58.745 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:12:58.746 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:12:58.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:12:58.746 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:58.746 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:12:58.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-13T19:12:58.812 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-13T19:12:58.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-13T19:12:58.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:58.813 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:12:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-13T19:12:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-13T19:12:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251' 2026-02-13T19:12:58.875 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:58.875 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:12:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-13T19:12:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-13T19:12:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251 2-77309411335' 2026-02-13T19:12:58.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:12:58.943 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:12:59.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-13T19:12:59.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-13T19:12:59.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251 2-77309411335 3-111669149702' 2026-02-13T19:12:59.006 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:12:59.006 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-13T19:12:59.007 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:12:59.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:12:59.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-13T19:12:59.008 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:12:59.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-13T19:12:59.009 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-13T19:12:59.009 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803784 2026-02-13T19:12:59.009 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:12:59.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-13T19:12:59.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:00.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:00.238 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:00.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-13T19:13:00.469 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:00.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-163208757251 2026-02-13T19:13:00.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:00.471 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:00.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-163208757251 2026-02-13T19:13:00.472 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:00.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757251 2026-02-13T19:13:00.473 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 163208757251' 2026-02-13T19:13:00.473 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 163208757251 2026-02-13T19:13:00.473 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:00.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-13T19:13:00.698 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:00.698 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411335 2026-02-13T19:13:00.699 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:00.699 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:00.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-13T19:13:00.700 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:00.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-13T19:13:00.701 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-13T19:13:00.701 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 77309411335 2026-02-13T19:13:00.702 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:00.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-13T19:13:00.928 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:00.928 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-13T19:13:00.928 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:00.929 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:00.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-13T19:13:00.930 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:00.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-13T19:13:00.931 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-13T19:13:00.931 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149702 2026-02-13T19:13:00.931 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:01.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-13T19:13:01.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:01.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:01.159 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:01.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:01.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:01.471 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:01.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:01.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:01.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:01.694 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:02.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:02.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:02.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:02.010 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-94542-1 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-94542-1 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:13:02.011 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-94542-1 td/test-erasure-eio/COPY 2026-02-13T19:13:02.026 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:13:02.027 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:13:02.028 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-13T19:13:02.028 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-94542-1 td/test-erasure-eio 2 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-13T19:13:02.031 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-94542-1 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-94542-1 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-94542-1 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-94542-1 2026-02-13T19:13:02.032 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:02.271 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:02.271 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-94542-1 remove 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-13T19:13:02.272 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-94542-1 remove 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:02.273 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:02.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:02.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-94542-1 remove 2026-02-13T19:13:02.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-13T19:13:02.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-13T19:13:02.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-13T19:13:02.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-13T19:13:02.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:02.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-94542-1 remove 2026-02-13T19:13:02.674 INFO:tasks.workunit.client.0.trial034.stdout:remove 2#2:9c5f4564:::obj-remove-94542-1:head# 2026-02-13T19:13:02.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-13T19:13:02.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:02.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:13:02.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:13:02.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:13:02.726 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:02.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:start osd.2 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:02.728 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:13:02.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:13:02.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:13:02.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:13:02.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:13:02.740 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:02.739+0000 7f954f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:02.741 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:02.740+0000 7f954f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:02.742 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:02.741+0000 7f954f40d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:02.869 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:02.868+0000 7f954f40d780 -1 Falling back to public interface 2026-02-13T19:13:02.950 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:0 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:02.951 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:03.163 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:03.162+0000 7f954f40d780 -1 osd.2 40 log_to_monitors true 2026-02-13T19:13:03.180 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:04.181 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:04.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:04.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:04.182 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:04.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:04.182 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:04.411 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 up in weight 1 up_from 43 up_thru 35 down_at 40 last_clean_interval [18,40) [v2:127.0.0.1:6818/3390538697,v1:127.0.0.1:6819/3390538697] [v2:127.0.0.1:6820/3390538697,v1:127.0.0.1:6821/3390538697] exists,up 90aef19f-e528-4015-a943-76b8b45371d7 2026-02-13T19:13:04.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:04.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:13:04.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:13:04.500 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:13:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:13:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:13:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:13:04.501 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:13:04.501 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:13:04.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:13:04.729 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:04.730 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:13:04.730 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:13:04.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:13:04.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:04.730 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:13:04.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-13T19:13:04.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-13T19:13:04.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-13T19:13:04.795 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:04.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:13:04.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757254 2026-02-13T19:13:04.860 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757254 2026-02-13T19:13:04.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254' 2026-02-13T19:13:04.861 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:04.861 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:13:04.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-13T19:13:04.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-13T19:13:04.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254 2-184683593731' 2026-02-13T19:13:04.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:04.926 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:13:04.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-13T19:13:04.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-13T19:13:04.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254 2-184683593731 3-111669149705' 2026-02-13T19:13:04.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:04.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-13T19:13:04.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:04.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:13:04.991 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-13T19:13:04.991 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:04.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-13T19:13:04.992 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-13T19:13:04.992 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.0 seq 25769803787 2026-02-13T19:13:04.992 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:05.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-13T19:13:05.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:06.217 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:06.217 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:06.440 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-13T19:13:06.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:06.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-163208757254 2026-02-13T19:13:06.441 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:06.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:06.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-163208757254 2026-02-13T19:13:06.442 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:06.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757254 2026-02-13T19:13:06.444 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 163208757254' 2026-02-13T19:13:06.444 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.1 seq 163208757254 2026-02-13T19:13:06.444 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:06.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757254 -lt 163208757254 2026-02-13T19:13:06.674 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:06.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-184683593731 2026-02-13T19:13:06.675 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:06.675 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:06.676 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-13T19:13:06.676 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:06.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593731 2026-02-13T19:13:06.677 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-13T19:13:06.677 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.2 seq 184683593731 2026-02-13T19:13:06.678 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:06.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-13T19:13:06.907 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:06.907 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149705 2026-02-13T19:13:06.908 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:06.908 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:06.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-13T19:13:06.909 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:06.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149705 2026-02-13T19:13:06.910 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-13T19:13:06.910 INFO:tasks.workunit.client.0.trial034.stderr:waiting osd.3 seq 111669149705 2026-02-13T19:13:06.911 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:07.136 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-13T19:13:07.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:07.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:07.137 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:07.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:07.450 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:07.451 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:07.672 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:07.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:07.673 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:07.674 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:07.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:07.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:07.997 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-94542-1 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-94542-1 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:13:07.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-94542-1 td/test-erasure-eio/COPY 2026-02-13T19:13:08.014 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:13:08.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:13:08.015 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-13T19:13:08.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:13:08.016 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:13:08.017 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:13:08.288 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:13:08.296 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:13:08.586 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:08.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:08.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:08.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:08.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:08.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:08.704 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:08.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:08.705 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:08.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:08.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:08.707 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:08.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:08.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:08.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:08.709 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:08.709 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:08.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:13:08.710 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:08.720 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:08.721 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.721 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:08.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:08.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:08.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:08.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:08.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:08.724 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:08.725 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:08.725 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:08.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:08.726 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:08.727 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:08.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:08.727 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:08.728 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:08.728 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:08.729 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:08.730 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:13:08.731 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:08.731 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:08.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.732 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:08.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:08.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:08.733 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:13:08.733 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:13:08.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.734 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.734 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:13:08.736 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:13:08.737 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:13:08.754 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:13:08.754 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:08.754 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:08.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:08.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.755 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.755 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:08.755 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:13:08.778 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:13:08.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:13:08.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:13:08.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:13:08.779 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:13:08.781 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:13:08.825 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:13:08.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:13:08.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:13:08.826 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:13:08.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:13:08.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:13:08.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:08.827 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:08.827 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:08.828 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.828 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.828 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:08.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:13:08.828 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:13:08.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:13:08.871 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:08.973 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:08.974 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:13:08.974 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:13:08.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:13:08.989 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:13:09.085 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:13:09.092 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:13:10.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:13:10.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:13:10.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:13:10.135 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:13:10.139 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:13:10.141 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:13:08.771+0000 7f9b2da08c00 0 load: jerasure load: lrc 2026-02-13T19:13:10.142 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:13:10.143 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:13:10.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:10.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:13:10.149 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:10.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:10.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:10.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:10.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7a1668a-eb4c-4a0a-8248-53cc41105518 2026-02-13T19:13:10.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c7a1668a-eb4c-4a0a-8248-53cc41105518' 2026-02-13T19:13:10.154 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 c7a1668a-eb4c-4a0a-8248-53cc41105518 2026-02-13T19:13:10.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:10.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDGd49p/htGCRAAq2s3SEPe4zWSRSj9BVXD2g== 2026-02-13T19:13:10.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDGd49p/htGCRAAq2s3SEPe4zWSRSj9BVXD2g=="}' 2026-02-13T19:13:10.157 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7a1668a-eb4c-4a0a-8248-53cc41105518 -i td/test-erasure-eio/0/new.json 2026-02-13T19:13:10.258 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:10.266 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:13:10.267 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDGd49p/htGCRAAq2s3SEPe4zWSRSj9BVXD2g== --osd-uuid c7a1668a-eb4c-4a0a-8248-53cc41105518 2026-02-13T19:13:10.283 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.282+0000 7fca43662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.284 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.283+0000 7fca43662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.285 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.284+0000 7fca43662780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.286 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.284+0000 7fca43662780 -1 bdev(0x2a5ec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:10.286 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.284+0000 7fca43662780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:13:10.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:13:10.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:10.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:13:10.704 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:13:10.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:10.806 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:13:10.807 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:13:10.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:10.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:10.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:10.808 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:10.822 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.821+0000 7f374a859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.829 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.822+0000 7f374a859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.829 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:10.823+0000 7f374a859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:13:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:13:10.903 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:10.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:10.904 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:10.904 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:10.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:10.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:10.905 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:13:10.998 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:11.017 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:11.016+0000 7f374a859780 -1 Falling back to public interface 2026-02-13T19:13:11.268 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:11.267+0000 7f374a859780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:13:11.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:11.999 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:11.999 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:12.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:12.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:12.000 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:13:12.224 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3939474908,v1:127.0.0.1:6803/3939474908] [v2:127.0.0.1:6804/3939474908,v1:127.0.0.1:6805/3939474908] exists,up c7a1668a-eb4c-4a0a-8248-53cc41105518 2026-02-13T19:13:12.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:12.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:12.228 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:12.229 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:12.230 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:12.231 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2d6f7688-46e6-4634-a1f1-59cadd7b3674 2026-02-13T19:13:12.232 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 2d6f7688-46e6-4634-a1f1-59cadd7b3674 2026-02-13T19:13:12.232 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2d6f7688-46e6-4634-a1f1-59cadd7b3674' 2026-02-13T19:13:12.232 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:12.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDId49pUWEsDhAAQSORCgrSZF1HS7o4/kxt7Q== 2026-02-13T19:13:12.238 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDId49pUWEsDhAAQSORCgrSZF1HS7o4/kxt7Q=="}' 2026-02-13T19:13:12.239 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2d6f7688-46e6-4634-a1f1-59cadd7b3674 -i td/test-erasure-eio/1/new.json 2026-02-13T19:13:12.471 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:12.479 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:13:12.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDId49pUWEsDhAAQSORCgrSZF1HS7o4/kxt7Q== --osd-uuid 2d6f7688-46e6-4634-a1f1-59cadd7b3674 2026-02-13T19:13:12.497 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:12.496+0000 7f8562a56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:12.498 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:12.497+0000 7f8562a56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:12.499 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:12.498+0000 7f8562a56780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:12.500 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:12.499+0000 7f8562a56780 -1 bdev(0x2c90c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:12.500 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:12.499+0000 7f8562a56780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:13:12.821 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:13:12.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:12.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:13:12.822 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:13:12.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:13.140 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:13:13.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:13:13.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:13.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:13.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:13.141 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:13.155 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:13.153+0000 7f3c3ce32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:13.156 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:13.155+0000 7f3c3ce32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:13.157 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:13.156+0000 7f3c3ce32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:13.306 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:13.305+0000 7f3c3ce32780 -1 Falling back to public interface 2026-02-13T19:13:13.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:13:13.373 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:13.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:13:13.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:13.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:13.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:13.374 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:13.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:13.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:13.375 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:13:13.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:13.634 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:13.633+0000 7f3c3ce32780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:13:14.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:14.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:14.610 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:14.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:14.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:14.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:13:14.833 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3631128725,v1:127.0.0.1:6811/3631128725] [v2:127.0.0.1:6812/3631128725,v1:127.0.0.1:6813/3631128725] exists,up 2d6f7688-46e6-4634-a1f1-59cadd7b3674 2026-02-13T19:13:14.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:14.837 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:14.838 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:14.839 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:14.840 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4d813f55-703c-44b7-abc8-3fd7f9083ace 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4d813f55-703c-44b7-abc8-3fd7f9083ace' 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 4d813f55-703c-44b7-abc8-3fd7f9083ace 2026-02-13T19:13:14.841 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:14.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDKd49ptHxoMhAAs28m7cAtd9LASD83cPsWhA== 2026-02-13T19:13:14.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDKd49ptHxoMhAAs28m7cAtd9LASD83cPsWhA=="}' 2026-02-13T19:13:14.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4d813f55-703c-44b7-abc8-3fd7f9083ace -i td/test-erasure-eio/2/new.json 2026-02-13T19:13:15.078 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:13:15.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:13:15.086 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKd49ptHxoMhAAs28m7cAtd9LASD83cPsWhA== --osd-uuid 4d813f55-703c-44b7-abc8-3fd7f9083ace 2026-02-13T19:13:15.102 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.101+0000 7fec6802e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.103 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.102+0000 7fec6802e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.104 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.103+0000 7fec6802e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.104 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.103+0000 7fec6802e780 -1 bdev(0x4030c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:15.104 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.103+0000 7fec6802e780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:13:15.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:13:15.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:15.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:13:15.434 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:13:15.434 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:15.779 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:13:15.779 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:13:15.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:15.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:15.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:15.780 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:15.793 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.793+0000 7f0effa12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.795 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.794+0000 7f0effa12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.796 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.795+0000 7f0effa12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:15.930 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:15.929+0000 7f0effa12780 -1 Falling back to public interface 2026-02-13T19:13:16.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:16.005 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:16.006 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:16.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:16.007 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:16.233 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:16.268 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:16.267+0000 7f0effa12780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:17.235 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:17.465 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2735590093,v1:127.0.0.1:6819/2735590093] [v2:127.0.0.1:6820/2735590093,v1:127.0.0.1:6821/2735590093] exists,up 4d813f55-703c-44b7-abc8-3fd7f9083ace 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:17.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:17.467 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:17.468 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:13:17.469 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:17.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2efcfabc-ce8d-4799-a6ea-ced00fbeb12c 2026-02-13T19:13:17.470 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2efcfabc-ce8d-4799-a6ea-ced00fbeb12c' 2026-02-13T19:13:17.470 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 2efcfabc-ce8d-4799-a6ea-ced00fbeb12c 2026-02-13T19:13:17.470 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:17.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNd49pHuGTHBAACSTkiwi+WN9HNG5Sn4rr+Q== 2026-02-13T19:13:17.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNd49pHuGTHBAACSTkiwi+WN9HNG5Sn4rr+Q=="}' 2026-02-13T19:13:17.480 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2efcfabc-ce8d-4799-a6ea-ced00fbeb12c -i td/test-erasure-eio/3/new.json 2026-02-13T19:13:17.716 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:13:17.723 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:13:17.724 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNd49pHuGTHBAACSTkiwi+WN9HNG5Sn4rr+Q== --osd-uuid 2efcfabc-ce8d-4799-a6ea-ced00fbeb12c 2026-02-13T19:13:17.740 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:17.739+0000 7f293d02b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:17.741 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:17.741+0000 7f293d02b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:17.742 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:17.742+0000 7f293d02b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:17.743 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:17.742+0000 7f293d02b780 -1 bdev(0x2904c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:17.743 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:17.742+0000 7f293d02b780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:13:18.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:13:18.066 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:18.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:13:18.067 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:13:18.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:18.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:13:18.381 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:13:18.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:18.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:18.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:18.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:18.395 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:18.394+0000 7fbbed048780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:18.396 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:18.395+0000 7fbbed048780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:18.397 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:18.396+0000 7fbbed048780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:18.529 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:18.528+0000 7fbbed048780 -1 Falling back to public interface 2026-02-13T19:13:18.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:13:18.609 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:18.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:13:18.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:18.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:18.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:18.610 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:18.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:18.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:18.611 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:13:18.814 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:18.813+0000 7fbbed048780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:13:18.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:19.846 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:19.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:19.847 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:19.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:19.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:19.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:13:20.074 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2595388393,v1:127.0.0.1:6827/2595388393] [v2:127.0.0.1:6828/2595388393,v1:127.0.0.1:6829/2595388393] exists,up 2efcfabc-ce8d-4799-a6ea-ced00fbeb12c 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:13:20.075 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:20.076 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:20.076 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:20.076 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:13:20.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:13:20.076 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:13:20.120 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:13:20.123 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:13:20.126 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:13:11.018+0000 7f374a859780 0 load: jerasure load: lrc 2026-02-13T19:13:20.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-13T19:13:20.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:13:20.127 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:13:20.128 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:13:20.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:13:20.413 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:13:20.783 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:13:20.791 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:13:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:13:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:13:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:13:21.793 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:13:21.793 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:13:21.794 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:13:21.794 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:13:21.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:13:21.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:13:21.795 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:13:21.882 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:13:22.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:22.110 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:13:22.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:13:22.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:13:22.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:13:22.172 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:22.172 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:13:22.236 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-13T19:13:22.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-13T19:13:22.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-13T19:13:22.237 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:22.237 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:13:22.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-13T19:13:22.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-13T19:13:22.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-81604378628' 2026-02-13T19:13:22.303 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:22.303 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:13:22.366 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-13T19:13:22.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-13T19:13:22.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-81604378628 3-111669149700' 2026-02-13T19:13:22.367 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:22.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:13:22.367 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:22.368 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:13:22.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:13:22.369 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:22.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:13:22.370 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:13:22.370 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:13:22.370 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:22.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:13:22.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:23.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:23.599 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:23.822 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:13:23.823 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:23.823 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-13T19:13:23.823 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:23.824 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:23.825 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-13T19:13:23.825 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:23.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-13T19:13:23.826 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-13T19:13:23.826 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574852 2026-02-13T19:13:23.827 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:24.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-13T19:13:24.052 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:24.052 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378628 2026-02-13T19:13:24.053 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:24.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:24.054 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-13T19:13:24.054 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:24.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378628 2026-02-13T19:13:24.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-13T19:13:24.055 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 81604378628 2026-02-13T19:13:24.056 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:24.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378628 2026-02-13T19:13:24.286 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:24.286 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149700 2026-02-13T19:13:24.287 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:24.287 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:24.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-13T19:13:24.288 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:24.289 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149700 2026-02-13T19:13:24.289 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149700 2026-02-13T19:13:24.290 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-13T19:13:24.290 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:24.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-13T19:13:24.528 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:24.528 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:24.528 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:24.847 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:24.848 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:24.848 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:25.073 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:25.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:25.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:25.074 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-94542-0 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-94542-0 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:13:25.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:25.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:13:25.394 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-94542-0 td/test-erasure-eio/ORIGINAL 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-0 td/test-erasure-eio 0 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-0 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-13T19:13:25.411 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:13:25.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:25.412 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-0 2026-02-13T19:13:25.412 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.412 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:25.642 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:13:25.643 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-13T19:13:25.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:13:25.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-13T19:13:25.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:13:25.644 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-13T19:13:25.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:13:25.645 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:13:25.645 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:13:25.646 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:13:25.693 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:13:25.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:13:25.694 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:13:25.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-13T19:13:25.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.3.asok 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:13:25.696 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-94542-0 0 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-0 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:13:25.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-94542-0 td/test-erasure-eio/COPY 2026-02-13T19:13:25.760 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:13:25.761 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:13:25.762 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-13T19:13:25.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.762 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:25.763 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-0 2026-02-13T19:13:25.763 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-0 2026-02-13T19:13:25.763 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:13:25.990 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:25.991 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:25.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:25.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:25.991 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:26.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:26.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-13T19:13:26.395 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already out. 2026-02-13T19:13:26.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-94542-0 2026-02-13T19:13:26.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-13T19:13:26.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:26.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-0 2026-02-13T19:13:26.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-0 2026-02-13T19:13:26.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:26.631 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:26.632 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:26.632 INFO:tasks.workunit.client.0.trial034.stderr:0' 2026-02-13T19:13:26.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-13T19:13:26.632 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-13T19:13:26.912 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already in. 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:26.923 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:26.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:13:26.925 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:13:26.926 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:13:26.926 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:26.927 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:13:26.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:13:26.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:13:26.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:13:26.927 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:13:26.939 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:26.939+0000 7f4ed883a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:26.941 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:26.940+0000 7f4ed883a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:26.942 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:26.941+0000 7f4ed883a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:27.102 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:27.102+0000 7f4ed883a780 -1 Falling back to public interface 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:27.153 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:27.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:27.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:27.378 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:27.400 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:27.399+0000 7f4ed883a780 -1 osd.2 33 log_to_monitors true 2026-02-13T19:13:28.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:28.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:28.380 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:28.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:28.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:28.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:28.605 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 42 up_thru 19 down_at 34 last_clean_interval [19,33) [v2:127.0.0.1:6818/3912566008,v1:127.0.0.1:6819/3912566008] [v2:127.0.0.1:6820/3912566008,v1:127.0.0.1:6821/3912566008] exists,up 4d813f55-703c-44b7-abc8-3fd7f9083ace 2026-02-13T19:13:28.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:28.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:28.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:28.607 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:13:28.608 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:13:28.695 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:28.919 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:13:28.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-13T19:13:28.981 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-13T19:13:28.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-13T19:13:28.982 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:28.982 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:13:29.043 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-13T19:13:29.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-13T19:13:29.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-13T19:13:29.044 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:29.044 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:13:29.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-13T19:13:29.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-13T19:13:29.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-180388626435' 2026-02-13T19:13:29.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:29.107 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149703 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149703 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-180388626435 3-111669149703' 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-13T19:13:29.173 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:29.174 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:13:29.174 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-13T19:13:29.175 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:29.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-13T19:13:29.176 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803784 2026-02-13T19:13:29.176 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-13T19:13:29.176 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:29.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-13T19:13:29.403 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:29.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-13T19:13:29.404 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:29.404 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:29.405 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-13T19:13:29.405 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:29.406 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-13T19:13:29.406 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574855 2026-02-13T19:13:29.407 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-13T19:13:29.407 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:29.636 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574855 2026-02-13T19:13:29.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:30.637 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:30.638 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:30.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574855 2026-02-13T19:13:30.862 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:31.863 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:13:31.863 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:32.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574855 2026-02-13T19:13:32.093 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:32.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-180388626435 2026-02-13T19:13:32.094 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:32.095 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:32.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-180388626435 2026-02-13T19:13:32.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:32.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626435 2026-02-13T19:13:32.097 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-13T19:13:32.097 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 180388626435 2026-02-13T19:13:32.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:32.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-13T19:13:32.325 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:32.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149703 2026-02-13T19:13:32.326 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:32.326 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:32.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149703 2026-02-13T19:13:32.327 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:32.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149703 2026-02-13T19:13:32.328 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-13T19:13:32.329 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149703 2026-02-13T19:13:32.329 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:32.557 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149704 -lt 111669149703 2026-02-13T19:13:32.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:32.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:32.558 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:32.870 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:32.871 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:33.088 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:33.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:33.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:33.089 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:33.399 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:33.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:33.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:33.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:13:33.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:13:33.400 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:13:33.668 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:13:33.676 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:13:33.959 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:13:33.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:13:33.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:33.968 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:13:33.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:33.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:33.969 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:33.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:33.969 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:33.970 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:34.077 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:34.078 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:34.079 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:34.079 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:34.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:34.081 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:34.081 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:34.082 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:34.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:34.083 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:34.083 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:34.084 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:34.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:13:34.085 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:34.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:34.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.096 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.096 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:34.098 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:34.099 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:34.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:34.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:34.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:34.099 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:34.100 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:34.100 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:34.101 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:34.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:34.102 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:34.102 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:34.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:34.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:34.103 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:34.103 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:34.104 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:34.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:13:34.105 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:34.106 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:34.106 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.107 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:34.107 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:34.108 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-13T19:13:34.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-13T19:13:34.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.109 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.109 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.94542 2026-02-13T19:13:34.110 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-13T19:13:34.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-13T19:13:34.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-13T19:13:34.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-13T19:13:34.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-13T19:13:34.111 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-13T19:13:34.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-13T19:13:34.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-13T19:13:34.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-13T19:13:34.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-13T19:13:34.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:34.130 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-13T19:13:34.154 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-13T19:13:34.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-13T19:13:34.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:13:34.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:13:34.155 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-13T19:13:34.156 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-13T19:13:34.156 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:13:34.156 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:34.156 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:13:34.157 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get fsid 2026-02-13T19:13:34.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-13T19:13:34.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-13T19:13:34.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-13T19:13:34.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-13T19:13:34.198 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-13T19:13:34.199 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-13T19:13:34.199 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:34.199 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-13T19:13:34.200 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.94542/ceph-mon.a.asok config get mon_host 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-13T19:13:34.242 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-13T19:13:34.243 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:34.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:34.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:13:34.344 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-13T19:13:34.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-13T19:13:34.360 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-13T19:13:34.455 INFO:tasks.workunit.client.0.trial034.stderr:pool 'rbd' already exists 2026-02-13T19:13:34.462 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:35.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-mon.a.asok 2026-02-13T19:13:35.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-13T19:13:35.466 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-mon.a.asok log flush 2026-02-13T19:13:35.504 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:13:35.509 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-13T19:13:35.510 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:13:34.145+0000 7f6d74417c00 0 load: jerasure load: lrc 2026-02-13T19:13:35.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-13T19:13:35.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-13T19:13:35.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-13T19:13:35.511 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-13T19:13:35.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-13T19:13:35.512 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-13T19:13:35.512 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-13T19:13:35.512 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-13T19:13:35.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:35.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-13T19:13:35.516 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-13T19:13:35.517 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:35.518 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-13T19:13:35.519 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:35.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=69ef5fb0-064b-44cd-8bd5-df69a31fb743 2026-02-13T19:13:35.520 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 69ef5fb0-064b-44cd-8bd5-df69a31fb743' 2026-02-13T19:13:35.520 INFO:tasks.workunit.client.0.trial034.stdout:add osd0 69ef5fb0-064b-44cd-8bd5-df69a31fb743 2026-02-13T19:13:35.521 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:35.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDfd49pzBpbHxAAiu+++X5hkSFCyLt3SSBpGA== 2026-02-13T19:13:35.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDfd49pzBpbHxAAiu+++X5hkSFCyLt3SSBpGA=="}' 2026-02-13T19:13:35.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 69ef5fb0-064b-44cd-8bd5-df69a31fb743 -i td/test-erasure-eio/0/new.json 2026-02-13T19:13:35.627 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:35.633 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-13T19:13:35.634 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDfd49pzBpbHxAAiu+++X5hkSFCyLt3SSBpGA== --osd-uuid 69ef5fb0-064b-44cd-8bd5-df69a31fb743 2026-02-13T19:13:35.649 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:35.647+0000 7fa8d47c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:35.651 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:35.649+0000 7fa8d47c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:35.651 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:35.650+0000 7fa8d47c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:35.652 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:35.650+0000 7fa8d47c6780 -1 bdev(0x2c18c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:35.652 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:35.650+0000 7fa8d47c6780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-13T19:13:36.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-13T19:13:36.053 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:36.054 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-13T19:13:36.054 INFO:tasks.workunit.client.0.trial034.stdout:adding osd0 key to auth repository 2026-02-13T19:13:36.055 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:36.158 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-13T19:13:36.159 INFO:tasks.workunit.client.0.trial034.stdout:start osd.0 2026-02-13T19:13:36.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:36.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:36.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:36.159 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:36.174 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:36.172+0000 7fcf67060780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:36.176 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:36.174+0000 7fcf67060780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:36.177 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:36.175+0000 7fcf67060780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:36.255 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:36.256 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:36.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:36.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:36.257 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:13:36.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:36.372 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:36.371+0000 7fcf67060780 -1 Falling back to public interface 2026-02-13T19:13:36.629 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:36.627+0000 7fcf67060780 -1 osd.0 0 log_to_monitors true 2026-02-13T19:13:37.357 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:37.358 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:37.358 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:37.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:37.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:37.359 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-13T19:13:37.595 INFO:tasks.workunit.client.0.trial034.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/279618222,v1:127.0.0.1:6803/279618222] [v2:127.0.0.1:6804/279618222,v1:127.0.0.1:6805/279618222] exists,up 69ef5fb0-064b-44cd-8bd5-df69a31fb743 2026-02-13T19:13:37.598 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-13T19:13:37.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:37.600 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:37.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:37.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:37.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-13T19:13:37.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:37.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=331e8f93-4146-422d-9179-af6e56d9fb30 2026-02-13T19:13:37.602 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 331e8f93-4146-422d-9179-af6e56d9fb30' 2026-02-13T19:13:37.603 INFO:tasks.workunit.client.0.trial034.stdout:add osd1 331e8f93-4146-422d-9179-af6e56d9fb30 2026-02-13T19:13:37.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:37.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDhd49pj2RFJBAAz5D4pOGvEbW8TkrZuGVNeg== 2026-02-13T19:13:37.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDhd49pj2RFJBAAz5D4pOGvEbW8TkrZuGVNeg=="}' 2026-02-13T19:13:37.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 331e8f93-4146-422d-9179-af6e56d9fb30 -i td/test-erasure-eio/1/new.json 2026-02-13T19:13:37.839 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:37.847 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-13T19:13:37.848 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDhd49pj2RFJBAAz5D4pOGvEbW8TkrZuGVNeg== --osd-uuid 331e8f93-4146-422d-9179-af6e56d9fb30 2026-02-13T19:13:37.864 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:37.862+0000 7fcf5cc4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:37.865 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:37.863+0000 7fcf5cc4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:37.866 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:37.864+0000 7fcf5cc4a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:37.866 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:37.864+0000 7fcf5cc4a780 -1 bdev(0x3322c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:37.866 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:37.864+0000 7fcf5cc4a780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-13T19:13:38.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-13T19:13:38.188 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:38.189 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-13T19:13:38.189 INFO:tasks.workunit.client.0.trial034.stdout:adding osd1 key to auth repository 2026-02-13T19:13:38.190 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:38.513 INFO:tasks.workunit.client.0.trial034.stdout:start osd.1 2026-02-13T19:13:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-13T19:13:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:38.514 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:38.529 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:38.527+0000 7fdaf2612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:38.530 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:38.528+0000 7fdaf2612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:38.531 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:38.529+0000 7fdaf2612780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:38.677 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:38.675+0000 7fdaf2612780 -1 Falling back to public interface 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:38.742 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:38.743 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:38.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:38.743 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:13:38.967 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:38.966+0000 7fdaf2612780 -1 osd.1 0 log_to_monitors true 2026-02-13T19:13:38.971 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:39.973 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-13T19:13:40.202 INFO:tasks.workunit.client.0.trial034.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/960283764,v1:127.0.0.1:6811/960283764] [v2:127.0.0.1:6812/960283764,v1:127.0.0.1:6813/960283764] exists,up 331e8f93-4146-422d-9179-af6e56d9fb30 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-13T19:13:40.206 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:13:40.207 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:40.208 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:40.209 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1904a560-af28-49f3-955a-07c3bd8e7684 2026-02-13T19:13:40.210 INFO:tasks.workunit.client.0.trial034.stdout:add osd2 1904a560-af28-49f3-955a-07c3bd8e7684 2026-02-13T19:13:40.210 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1904a560-af28-49f3-955a-07c3bd8e7684' 2026-02-13T19:13:40.210 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:40.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDkd49pCwvWDBAANy5um6uRN55Xv7gQb0yxtw== 2026-02-13T19:13:40.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDkd49pCwvWDBAANy5um6uRN55Xv7gQb0yxtw=="}' 2026-02-13T19:13:40.216 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1904a560-af28-49f3-955a-07c3bd8e7684 -i td/test-erasure-eio/2/new.json 2026-02-13T19:13:40.449 INFO:tasks.workunit.client.0.trial034.stdout:2 2026-02-13T19:13:40.456 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-13T19:13:40.457 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkd49pCwvWDBAANy5um6uRN55Xv7gQb0yxtw== --osd-uuid 1904a560-af28-49f3-955a-07c3bd8e7684 2026-02-13T19:13:40.473 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:40.471+0000 7f072ce2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:40.474 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:40.472+0000 7f072ce2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:40.475 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:40.473+0000 7f072ce2b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:40.475 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:40.473+0000 7f072ce2b780 -1 bdev(0x3b50c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:40.475 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:40.473+0000 7f072ce2b780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-13T19:13:40.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-13T19:13:40.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:40.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-13T19:13:40.801 INFO:tasks.workunit.client.0.trial034.stdout:adding osd2 key to auth repository 2026-02-13T19:13:40.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:41.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-13T19:13:41.125 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:13:41.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:41.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:41.125 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:41.126 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:41.139 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:41.138+0000 7faf98c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:41.141 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:41.139+0000 7faf98c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:41.142 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:41.140+0000 7faf98c2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:41.329 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:41.328+0000 7faf98c2f780 -1 Falling back to public interface 2026-02-13T19:13:41.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-13T19:13:41.347 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:41.348 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:41.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:41.349 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:41.573 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:41.614 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:41.612+0000 7faf98c2f780 -1 osd.2 0 log_to_monitors true 2026-02-13T19:13:42.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:42.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:42.575 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:42.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:42.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:42.576 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:42.799 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3272907914,v1:127.0.0.1:6819/3272907914] [v2:127.0.0.1:6820/3272907914,v1:127.0.0.1:6821/3272907914] exists,up 1904a560-af28-49f3-955a-07c3bd8e7684 2026-02-13T19:13:42.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:42.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:42.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:42.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:42.802 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:42.802 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:42.802 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:42.802 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-13T19:13:42.803 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-13T19:13:42.804 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=75b2c106-82a8-49cc-ba2f-2b279fdd3c32 2026-02-13T19:13:42.804 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 75b2c106-82a8-49cc-ba2f-2b279fdd3c32' 2026-02-13T19:13:42.804 INFO:tasks.workunit.client.0.trial034.stdout:add osd3 75b2c106-82a8-49cc-ba2f-2b279fdd3c32 2026-02-13T19:13:42.804 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-13T19:13:42.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDmd49pYFJxMBAAT3xH/P2bQ4lUzFLWgMP5xg== 2026-02-13T19:13:42.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDmd49pYFJxMBAAT3xH/P2bQ4lUzFLWgMP5xg=="}' 2026-02-13T19:13:42.813 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 75b2c106-82a8-49cc-ba2f-2b279fdd3c32 -i td/test-erasure-eio/3/new.json 2026-02-13T19:13:43.039 INFO:tasks.workunit.client.0.trial034.stdout:3 2026-02-13T19:13:43.047 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-13T19:13:43.048 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDmd49pYFJxMBAAT3xH/P2bQ4lUzFLWgMP5xg== --osd-uuid 75b2c106-82a8-49cc-ba2f-2b279fdd3c32 2026-02-13T19:13:43.063 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.061+0000 7f8b12237780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.065 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.063+0000 7f8b12237780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.065 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.064+0000 7f8b12237780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.066 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.064+0000 7f8b12237780 -1 bdev(0x3c6ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-13T19:13:43.066 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.064+0000 7f8b12237780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-13T19:13:43.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-13T19:13:43.384 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-13T19:13:43.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-13T19:13:43.385 INFO:tasks.workunit.client.0.trial034.stdout:adding osd3 key to auth repository 2026-02-13T19:13:43.386 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-13T19:13:43.706 INFO:tasks.workunit.client.0.trial034.stdout:start osd.3 2026-02-13T19:13:43.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-13T19:13:43.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:43.707 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-13T19:13:43.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-13T19:13:43.708 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-13T19:13:43.721 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.719+0000 7fd123c39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.722 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.720+0000 7fd123c39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.723 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.721+0000 7fd123c39780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:43.866 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:43.864+0000 7fd123c39780 -1 Falling back to public interface 2026-02-13T19:13:43.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-13T19:13:43.940 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:43.941 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:43.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:43.942 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:13:44.163 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:44.222 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:44.220+0000 7fd123c39780 -1 osd.3 0 log_to_monitors true 2026-02-13T19:13:45.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:45.164 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:45.165 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:45.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:45.165 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:45.166 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-13T19:13:45.389 INFO:tasks.workunit.client.0.trial034.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1529374031,v1:127.0.0.1:6827/1529374031] [v2:127.0.0.1:6828/1529374031,v1:127.0.0.1:6829/1529374031] exists,up 75b2c106-82a8-49cc-ba2f-2b279fdd3c32 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-13T19:13:45.391 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:45.392 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:45.392 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:45.392 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.0.asok 2026-02-13T19:13:45.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-13T19:13:45.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.0.asok log flush 2026-02-13T19:13:45.434 INFO:tasks.workunit.client.0.trial034.stdout:{} 2026-02-13T19:13:45.438 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-13T19:13:45.440 INFO:tasks.workunit.client.0.trial034.stdout:2026-02-13T19:13:36.373+0000 7fcf67060780 0 load: jerasure load: lrc 2026-02-13T19:13:45.441 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-13T19:13:45.442 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-13T19:13:45.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-13T19:13:45.732 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-13T19:13:46.105 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' already exists 2026-02-13T19:13:46.112 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-13T19:13:47.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-13T19:13:47.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:13:47.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:13:47.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:13:47.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:13:47.115 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:13:47.116 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:13:47.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:13:47.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:13:47.116 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:13:47.205 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:13:47.432 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:13:47.432 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:47.433 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:13:47.433 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:13:47.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:13:47.433 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:47.433 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:13:47.498 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-13T19:13:47.499 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-13T19:13:47.499 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-13T19:13:47.499 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:47.499 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:13:47.561 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-13T19:13:47.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-13T19:13:47.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-13T19:13:47.562 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:47.562 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:13:47.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-13T19:13:47.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-13T19:13:47.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411331' 2026-02-13T19:13:47.625 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:47.626 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:13:47.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-13T19:13:47.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-13T19:13:47.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411331 3-111669149699' 2026-02-13T19:13:47.689 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:47.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-13T19:13:47.690 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:47.690 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:13:47.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-13T19:13:47.691 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:47.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-13T19:13:47.692 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-13T19:13:47.692 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803780 2026-02-13T19:13:47.693 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:47.922 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-13T19:13:47.922 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:48.924 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:48.924 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:49.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-13T19:13:49.148 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:49.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-13T19:13:49.149 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:49.150 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:49.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-13T19:13:49.151 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:49.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-13T19:13:49.152 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-13T19:13:49.152 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574852 2026-02-13T19:13:49.152 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:49.379 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-13T19:13:49.380 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:49.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-13T19:13:49.380 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:49.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:49.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-13T19:13:49.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:49.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-13T19:13:49.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-13T19:13:49.383 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 77309411331 2026-02-13T19:13:49.384 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:49.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-13T19:13:49.608 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:49.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-13T19:13:49.609 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:49.610 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:49.610 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-13T19:13:49.610 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:49.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-13T19:13:49.612 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-13T19:13:49.612 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149699 2026-02-13T19:13:49.612 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:49.835 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-13T19:13:49.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:49.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:49.836 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:50.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:50.153 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:50.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:50.153 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:50.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:50.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:50.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:50.154 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:50.374 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:50.374 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:50.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:50.375 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:50.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:50.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:50.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:50.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-13T19:13:50.703 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-94542-1 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-94542-1 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-13T19:13:50.704 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-94542-1 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-13T19:13:50.705 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-94542-1 td/test-erasure-eio/ORIGINAL 2026-02-13T19:13:50.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-94542-1 td/test-erasure-eio 1 2026-02-13T19:13:50.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-13T19:13:50.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-13T19:13:50.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-13T19:13:50.721 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-94542-1 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-94542-1 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-1 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-1 2026-02-13T19:13:50.722 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:50.951 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-13T19:13:50.952 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-13T19:13:50.954 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:13:50.955 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.94542/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-13T19:13:51.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-13T19:13:51.001 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:51.003 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.94542/ceph-osd.1.asok 2026-02-13T19:13:51.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-13T19:13:51.004 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.94542/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-94542-1 1 2026-02-13T19:13:51.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-94542-1 2026-02-13T19:13:51.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-13T19:13:51.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-13T19:13:51.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-94542-1 2026-02-13T19:13:51.050 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-13T19:13:51.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-13T19:13:51.051 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-94542-1 td/test-erasure-eio/COPY 2026-02-13T19:13:51.067 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-13T19:13:51.068 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-13T19:13:51.069 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-13T19:13:51.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-94542-1 2026-02-13T19:13:51.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:51.069 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-1 2026-02-13T19:13:51.070 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-1 2026-02-13T19:13:51.070 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr:2' 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-13T19:13:51.298 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:51.299 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:51.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:51.605 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-13T19:13:51.881 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already out. 2026-02-13T19:13:51.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-94542-1 2026-02-13T19:13:51.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-13T19:13:51.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-13T19:13:51.890 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-94542-1 2026-02-13T19:13:51.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-94542-1 2026-02-13T19:13:51.891 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-13T19:13:52.124 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-13T19:13:52.124 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:52.124 INFO:tasks.workunit.client.0.trial034.stderr:0' 2026-02-13T19:13:52.124 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-13T19:13:52.124 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-13T19:13:52.369 INFO:tasks.workunit.client.0.trial034.stderr:osd.2 is already in. 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-13T19:13:52.381 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:52.382 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-13T19:13:52.383 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-13T19:13:52.384 INFO:tasks.workunit.client.0.trial034.stdout:start osd.2 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=11670723-6ddc-48f5-be69-c48cc092d5d5 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.94542/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-13T19:13:52.385 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-13T19:13:52.399 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:52.398+0000 7f8b8c010780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:52.401 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:52.400+0000 7f8b8c010780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:52.402 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:52.401+0000 7f8b8c010780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-13T19:13:52.555 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:52.554+0000 7f8b8c010780 -1 Falling back to public interface 2026-02-13T19:13:52.613 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-13T19:13:52.614 INFO:tasks.workunit.client.0.trial034.stdout:0 2026-02-13T19:13:52.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:52.615 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:52.842 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-13T19:13:52.938 INFO:tasks.workunit.client.0.trial034.stderr:2026-02-13T19:13:52.937+0000 7f8b8c010780 -1 osd.2 34 log_to_monitors true 2026-02-13T19:13:53.843 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-13T19:13:53.844 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-13T19:13:53.844 INFO:tasks.workunit.client.0.trial034.stdout:1 2026-02-13T19:13:53.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-13T19:13:53.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-13T19:13:53.845 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-13T19:13:54.072 INFO:tasks.workunit.client.0.trial034.stdout:osd.2 up in weight 1 up_from 43 up_thru 19 down_at 35 last_clean_interval [18,34) [v2:127.0.0.1:6818/4076904625,v1:127.0.0.1:6819/4076904625] [v2:127.0.0.1:6820/4076904625,v1:127.0.0.1:6821/4076904625] exists,up 1904a560-af28-49f3-955a-07c3bd8e7684 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-13T19:13:54.074 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-13T19:13:54.075 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-13T19:13:54.162 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-13T19:13:54.392 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-13T19:13:54.392 INFO:tasks.workunit.client.0.trial034.stderr:1 2026-02-13T19:13:54.392 INFO:tasks.workunit.client.0.trial034.stderr:2 2026-02-13T19:13:54.392 INFO:tasks.workunit.client.0.trial034.stderr:3' 2026-02-13T19:13:54.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-13T19:13:54.393 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:54.393 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-13T19:13:54.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-13T19:13:54.459 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-13T19:13:54.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-13T19:13:54.460 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:54.460 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-13T19:13:54.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-13T19:13:54.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-13T19:13:54.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-13T19:13:54.527 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:54.527 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-13T19:13:54.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-13T19:13:54.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-13T19:13:54.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-184683593731' 2026-02-13T19:13:54.590 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-13T19:13:54.590 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-13T19:13:54.654 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-13T19:13:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-13T19:13:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-184683593731 3-111669149702' 2026-02-13T19:13:54.655 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:54.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-13T19:13:54.655 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:54.656 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-13T19:13:54.656 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-13T19:13:54.656 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:54.657 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-13T19:13:54.658 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.0 seq 25769803784 2026-02-13T19:13:54.658 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-13T19:13:54.658 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:54.881 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-13T19:13:54.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:55.882 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-13T19:13:55.883 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:56.113 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-13T19:13:56.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-13T19:13:57.114 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-13T19:13:57.115 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-13T19:13:57.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-13T19:13:57.342 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:57.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-13T19:13:57.343 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:57.343 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-13T19:13:57.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-13T19:13:57.344 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:57.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-13T19:13:57.345 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-13T19:13:57.346 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.1 seq 55834574855 2026-02-13T19:13:57.346 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-13T19:13:57.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574855 2026-02-13T19:13:57.571 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:57.572 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-184683593731 2026-02-13T19:13:57.572 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:57.572 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-13T19:13:57.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-13T19:13:57.573 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:57.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593731 2026-02-13T19:13:57.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-13T19:13:57.574 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.2 seq 184683593731 2026-02-13T19:13:57.575 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-13T19:13:57.798 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-13T19:13:57.799 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-13T19:13:57.799 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-13T19:13:57.799 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-13T19:13:57.800 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-13T19:13:57.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-13T19:13:57.801 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-13T19:13:57.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-13T19:13:57.802 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-13T19:13:57.802 INFO:tasks.workunit.client.0.trial034.stdout:waiting osd.3 seq 111669149702 2026-02-13T19:13:57.803 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-13T19:13:58.029 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-13T19:13:58.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-13T19:13:58.030 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:58.031 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:58.355 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-13T19:13:58.356 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-13T19:13:58.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-13T19:13:58.578 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-13T19:13:58.579 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-13T19:13:58.579 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-13T19:13:58.901 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-13T19:13:59.168 INFO:tasks.workunit.client.0.trial034.stderr:pool 'pool-jerasure' does not exist 2026-02-13T19:13:59.175 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-13T19:13:59.455 INFO:tasks.workunit.client.0.trial034.stderr:erasure-code-profile myprofile does not exist 2026-02-13T19:13:59.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-13T19:13:59.464 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:59.465 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:59.574 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:59.574 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:59.575 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:59.576 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:59.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:59.578 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:59.578 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:59.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:59.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:59.579 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:59.579 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:59.580 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:59.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-13T19:13:59.581 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:59.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:59.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:59.591 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:59.592 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:59.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:59.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:59.593 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-eio 0 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-13T19:13:59.594 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-13T19:13:59.595 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-13T19:13:59.596 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-13T19:13:59.596 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-13T19:13:59.597 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-13T19:13:59.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-13T19:13:59.597 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-13T19:13:59.598 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-13T19:13:59.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:59.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-13T19:13:59.599 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-13T19:13:59.599 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T19:13:59.600 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:13:59.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-13T19:13:59.601 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-13T19:13:59.602 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-13T19:13:59.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-13T19:13:59.603 INFO:tasks.workunit.client.0.trial034.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.94542 2026-02-13T19:13:59.603 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.94542 2026-02-13T19:13:59.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-13T19:13:59.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-13T19:13:59.604 INFO:tasks.workunit.client.0.trial034.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-13T19:13:59.605 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-13T19:13:59.605 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-13T19:13:59.670 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-13T19:13:59.670 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-13T19:14:00.035 DEBUG:teuthology.parallel:result is None 2026-02-13T19:14:00.036 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T19:14:00.061 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T19:14:00.062 DEBUG:teuthology.orchestra.run.trial034:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-13T19:14:00.120 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-13T19:14:00.120 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-13T19:14:00.135 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-13T19:14:00.136 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T19:14:00.296 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-13T19:14:00.307 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-13T19:14:00.308 DEBUG:teuthology.orchestra.run.trial034:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T19:14:00.324 INFO:teuthology.orchestra.run.trial034.stderr:bash: line 1: ntpq: command not found 2026-02-13T19:14:00.354 INFO:teuthology.orchestra.run.trial034.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T19:14:00.354 INFO:teuthology.orchestra.run.trial034.stdout:=============================================================================== 2026-02-13T19:14:00.355 INFO:teuthology.orchestra.run.trial034.stdout:^* chi3.us.ntp.li 2 6 377 11 +257us[ +267us] +/- 11ms 2026-02-13T19:14:00.355 INFO:teuthology.orchestra.run.trial034.stdout:^- ntp.maxhost.io 2 6 377 15 +624us[ +633us] +/- 55ms 2026-02-13T19:14:00.355 INFO:teuthology.orchestra.run.trial034.stdout:^- time.trtnw.net 2 6 377 13 +175us[ +184us] +/- 37ms 2026-02-13T19:14:00.355 INFO:teuthology.orchestra.run.trial034.stdout:^+ as393746.mci.trtnw.net 2 6 377 12 -540us[ -530us] +/- 23ms 2026-02-13T19:14:00.356 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-13T19:14:00.369 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-13T19:14:00.370 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-13T19:14:00.384 DEBUG:teuthology.orchestra.run.trial034:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T19:14:00.449 DEBUG:teuthology.orchestra.run.trial034:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-13T19:14:00.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:14:00.476 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-13T19:14:00.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-13T19:14:00.507 INFO:teuthology.task.internal:Duration was 1027.656386 seconds 2026-02-13T19:14:00.507 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-13T19:14:00.530 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-13T19:14:00.530 DEBUG:teuthology.orchestra.run.trial034:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T19:14:00.567 INFO:teuthology.orchestra.run.trial034.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T19:14:00.811 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-13T19:14:00.811 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial034.front.sepia.ceph.com 2026-02-13T19:14:00.811 DEBUG:teuthology.orchestra.run.trial034:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T19:14:00.868 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-13T19:14:00.868 DEBUG:teuthology.orchestra.run.trial034:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T19:14:00.965 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-13T19:14:00.965 DEBUG:teuthology.orchestra.run.trial034:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T19:14:00.988 INFO:teuthology.orchestra.run.trial034.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T19:14:00.988 INFO:teuthology.orchestra.run.trial034.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T19:14:00.988 INFO:teuthology.orchestra.run.trial034.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T19:14:00.988 INFO:teuthology.orchestra.run.trial034.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T19:14:00.988 INFO:teuthology.orchestra.run.trial034.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T19:14:00.998 INFO:teuthology.orchestra.run.trial034.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T19:14:00.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-13T19:14:01.011 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-13T19:14:01.011 DEBUG:teuthology.orchestra.run.trial034:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T19:14:01.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-13T19:14:01.076 DEBUG:teuthology.orchestra.run.trial034:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:14:01.127 INFO:teuthology.orchestra.run.trial034.stdout:kernel.core_pattern = core 2026-02-13T19:14:01.139 DEBUG:teuthology.orchestra.run.trial034:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T19:14:01.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T19:14:01.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-13T19:14:01.206 INFO:teuthology.task.internal:Transferring archived files... 2026-02-13T19:14:01.208 DEBUG:teuthology.misc:Transferring archived files from trial034:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49760/remote/trial034 2026-02-13T19:14:01.208 DEBUG:teuthology.orchestra.run.trial034:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T19:14:01.267 INFO:teuthology.task.internal:Removing archive directory... 2026-02-13T19:14:01.267 DEBUG:teuthology.orchestra.run.trial034:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T19:14:01.318 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-13T19:14:01.331 INFO:teuthology.task.internal:Not uploading archives. 2026-02-13T19:14:01.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-13T19:14:01.346 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-13T19:14:01.346 DEBUG:teuthology.orchestra.run.trial034:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T19:14:01.373 INFO:teuthology.orchestra.run.trial034.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 19:14 /home/ubuntu/cephtest 2026-02-13T19:14:01.375 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-13T19:14:01.392 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-13T19:14:01.421 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} duration: 1027.6563861370087 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-13T19:14:01.422 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T19:14:01.519 INFO:teuthology.run:pass